-- Logs begin at Mon 2022-05-23 14:39:50 UTC, end at Mon 2022-05-23 15:50:19 UTC. -- May 23 15:14:44.444106 dv-cinder-841125-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 23 15:14:44.455069 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 15:14:44 2022] *** May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: nodename: dv-cinder-841125-8-iscsi May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: machine: x86_64 May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: clock source: unix May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: pcre jit disabled May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: detected number of CPU cores: 4 May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: current working directory: / May 23 15:14:44.457057 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: detected binary path: /usr/bin/uwsgi-core May 23 15:14:44.457611 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: your processes number limit is 47680 May 23 15:14:44.457611 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: your memory page size is 4096 bytes May 23 15:14:44.457611 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: detected max file descriptor number: 2048 May 23 15:14:44.457611 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: lock engine: pthread robust mutexes May 23 15:14:44.457611 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: thunder lock: enabled May 23 15:14:44.457611 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 23 15:14:44.457611 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 15:14:44.501706 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: Python main interpreter initialized at 0x55e50e50f9c0 May 23 15:14:44.501706 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: python threads support enabled May 23 15:14:44.501706 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: your server socket listen backlog is limited to 100 connections May 23 15:14:44.501706 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: your mercy for graceful operations on workers is 90 seconds May 23 15:14:44.502019 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: mapped 403077 bytes (393 KB) for 2 cores May 23 15:14:44.502119 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: *** Operational MODE: preforking *** May 23 15:14:44.502389 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: *** uWSGI is running in multiple interpreter mode *** May 23 15:14:44.502389 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: spawned uWSGI master process (pid: 80411) May 23 15:14:44.502573 dv-cinder-841125-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 23 15:14:44.508912 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: spawned uWSGI worker 1 (pid: 80421, cores: 1) May 23 15:14:44.508912 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: spawned uWSGI worker 2 (pid: 80422, cores: 1) May 23 15:14:44.508912 dv-cinder-841125-8-iscsi devstack@keystone.service[80411]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 15:14:50.279812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:14:50.279812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: from cryptography.utils import int_from_bytes May 23 15:14:50.281195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:14:50.281195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: from cryptography.utils import int_from_bytes May 23 15:14:50.298660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:14:50.298660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: from cryptography.utils import int_from_bytes May 23 15:14:50.299132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:14:50.299132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: from cryptography.utils import int_from_bytes May 23 15:14:50.333291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:14:50.333489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:14:50.333683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] command line args: [] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:14:50.333957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:14:50.334137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ================================================================================ {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:14:50.334395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] admin_token = **** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.334644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] config_dir = [] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.334834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.335016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] config_source = [] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.335196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] debug = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.335395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.336790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.336790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.336790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.336790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.336790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] list_limit = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.337165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] log_config_append = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.337165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.337300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] log_dir = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.337490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] log_file = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.337752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.337993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.338245 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.338495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.338676 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.338889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.339144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.339416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.339677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.339936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.340196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.340452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.340737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.341033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.341353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.341872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.342078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] publish_errors = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.342342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.343132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.343312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.343552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.343791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.344029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] standard_threads = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.344264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.344510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.344686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.344875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] use_journal = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.345065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] use_json = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.345241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] use_stderr = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.345425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] use_syslog = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.345669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.345949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.346219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.346514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.346760 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.347014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.347096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:14:50.347270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.347356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:14:50.347432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] command line args: [] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:14:50.347507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.347716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:14:50.347792 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.347896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ================================================================================ {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:14:50.347977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.348152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] admin_token = **** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.348233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.348402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] config_dir = [] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.348477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.348591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.348670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.348775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] config_source = [] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.348953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.349046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] debug = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.349175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.349277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.349413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.349488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.349593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.349669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.349845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.349993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.350100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.350192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.350334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] list_limit = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.350416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.350513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] log_config_append = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.350623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.350710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.350854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] log_dir = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.350928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.351030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] log_file = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.351104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.351266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.351351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.351515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.351589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.351763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.351838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.352009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.352093 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.352187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.352268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.352364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.352499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.352608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.352738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.352856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.353032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.353117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.353284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.353367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.353517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.353598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.353749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.353823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.353998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.354149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.354248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.354324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.354488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.354561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.354738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.354969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.355051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.355192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] publish_errors = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.355272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.355429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.355511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.355659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.355739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.355837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.355912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.356088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.356164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.356326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.356401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.356562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] standard_threads = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.356635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.356795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.356907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.357057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.357138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.357260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.357343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.357422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] use_journal = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.357521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.357606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] use_json = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.357758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.357834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] use_stderr = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.357932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] use_syslog = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.358013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.358763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:14:50.358853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.359061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.359172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.359423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.359508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.359674 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.359748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.359917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.359997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.360157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.360312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.360384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.360452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.360616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.360690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.360884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.360978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.361203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.361369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.361443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.361604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.361687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.361856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.361937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.362095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.362170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.362329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.362412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.362572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.362648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.362812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.362888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.363052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.363140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.363309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.363382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.363561 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.363635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.363809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.363890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.364046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.364120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.364287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.364353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.364528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.364594 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.364779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.364844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.365094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.365173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.365285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.365354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.365518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.365602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.365754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.365829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.365990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.366065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.366229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.366305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.366467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.366546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.366705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.366786 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.366940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.367018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.367191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.367274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.367433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.367518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.367679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.367769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.367925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.368001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.368158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.368236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.368394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.368471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.368629 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.368703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.368859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.368962 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.369114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.369193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.369345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.369419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.369575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.369661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.369803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.369886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.370046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.370122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.370278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.370361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.370517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.370599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.370751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.370827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.370995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.371114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.371236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.371323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.371486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.371562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.371718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.371801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] auth.external = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.371949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.372024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.372187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.372263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.372427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.372578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.372656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.372726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] auth.password = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.372920 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.373001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] auth.token = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.373146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.373232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.373390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.373473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.373627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.373702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.373889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.374123 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.374201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.374390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.374732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.374897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.374972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.375131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.375219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.375372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.375450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] credential.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.375611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.375687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.375873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.375948 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.376113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.376190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.376354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.376430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.376602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.376678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.376840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.376972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.377102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.377179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.377340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.377424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.377591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.377667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.377833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.377908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.378073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.378157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.378321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.378398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.378560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.378636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.378800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.378884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.379038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.379131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.379354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] federation.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.379515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.379592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.379778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.379866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.380024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.380100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.380255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.380331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.380496 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.380583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.380733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.380809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.380995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.381086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.381237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.381322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.381475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.381552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.381707 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.381784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.381939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.382032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.382170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.382407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.382482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.382638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.382736 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.382880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.382989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.383118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.383231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.383370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.383471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.383609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.383722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.383867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.383943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.384130 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.384210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.384378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.384456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.384613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] auth.external = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.384690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.384847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.384940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.385144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.385222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.385381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.385464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.385613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] auth.password = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.385689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.385846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] auth.token = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.385926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.386088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.386164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.386323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.386399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.386572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.386652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.386804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.386882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.387055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.387150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.387305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.387394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.387536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.387613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.387773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.387848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] credential.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.388009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.388087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.388247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.388330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.388492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.388575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.388734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.388812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.389013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.389092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.389246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.389322 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.389542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.389634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.389791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.389876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.390031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.390117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.390291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.390368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.390580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.390657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.390817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.390900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.391060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.391138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.391293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.391370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.391533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.391618 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.391773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.391850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] federation.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.392019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.392096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.392259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.392344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.392499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.392577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.392729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.392805 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.393005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.393094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.393242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.393318 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.393475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.393552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.393724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.393808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.393972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.394048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.394210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.394288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.394446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.394530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.394685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.394762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395235 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.395952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.396102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.396178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.396343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.396420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.396578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.396662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.396810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.396924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.397074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.397152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.397303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.397410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.397536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.397619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.397775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.397858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.398024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.398110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.398272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.398350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.398519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.398596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.398767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.398852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.399014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.399091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.399246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.399324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.399493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.399577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.399727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.399844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.399957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.400030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.400186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.400271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.400421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.400497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.400663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.400739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.400931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.401025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.401173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.401251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.401403 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.401478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.401652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.401737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.401900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.401978 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.402137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.402213 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.402376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.402455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.402617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.402694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.402847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.402922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.403087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.403166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.403317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.403395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.403555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.403630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.403793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.403871 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] resource.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.404036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.404140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.404273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.404394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.404518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.404633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.404770 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.404904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.405597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.405795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.405887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.406056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.406132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.406309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.406385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.406551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.406638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.406791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.406868 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.407028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] role.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.407104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.407265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.407349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] role.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.407499 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.407575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.407743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.407820 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.407997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.408082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.408228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.408304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.408479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.408555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.408726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.408809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.408997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.409076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.409248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.409324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.409499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.409588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.409736 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.409804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.410008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.410075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.410285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.410428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.410541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.410756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.410823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.411013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.411217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.411293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.411472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.411540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.411727 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.411794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.411981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.412055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.412215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.412282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.412464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.412531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.412700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.412773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.412960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.413030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.413204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.413272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.413444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.413514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.413689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.413757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.413926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.413993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.414159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.414228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.414414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.414481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.414645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.414711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.414897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.414966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.415135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.415201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.415371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.415437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.415605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.415673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.415852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.415918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] token.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.416090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.416156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.416321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.416394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.416559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.416626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.416801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] resource.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.416908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.417080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.417193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.417320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.417400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.417555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.417622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.417806 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.417875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.418054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.418121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.418289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.418356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.418527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.418595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.418764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.418829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.419005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.419072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.419241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] role.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.419310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.419471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.419541 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] role.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.419708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.419774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.419960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.420028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.420247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.connection = **** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.420325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.420494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.420562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.421381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.421593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.421664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.421861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.422017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.422085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.422281 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.422361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.422543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.422611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.422791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.422858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.423034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.423108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.423272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.423339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.423520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.423587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.423759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.423835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.424011 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.424090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.424249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.424411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.424489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.424655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.424725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.424935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.425007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.425193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.425261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.425362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:14:50.425431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.425689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.425929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.426189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.426448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.426702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.426939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.427177 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.427414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.427669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.427931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.428183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.428423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.428664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.428917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.429160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] token.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.429400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.429650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.429889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.430148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.430396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.430637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.430885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.431131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.431382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.431621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.431861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.432099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.432347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.432597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.432841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.433115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.433412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.433675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.connection = **** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.434035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.434181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.434424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.434668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.434912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.435150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.435390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.435633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.435881 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.436121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.436401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.436817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.437102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.437344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.437592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.437825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.438070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.438310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:14:50.438904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:14:50.461738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.461970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.462201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.464278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.464513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.464739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.471590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.475456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.500848 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.501094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.502702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.502936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.554565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.555001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.606306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.606530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.607975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.607975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.673832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.673988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.674059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.674154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.674273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.674356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.674494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.674573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.674714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.674796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.674929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.675008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.698145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.698384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.698616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.698696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.698843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.698923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.699066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.699155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.699286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.699367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.699504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.699591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.699745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.699964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.711722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.711891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.724961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.725137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.725223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.725372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.725453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.725601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.725695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.725826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.725907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.726040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.726119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.726254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.726343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.726469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:14:50.743757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.743863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.743970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:14:50.744066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:14:50.744307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.744388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.744510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:14:50.744590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:14:50.744783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.744888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.745070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.745150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.745336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.745415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.745590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.745671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.757125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.757403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.757670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.757949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.758760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.759038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.759307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.759587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.766663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.768233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.769969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.770253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.770532 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.770811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.771091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.771349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.771539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.771621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.771817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.771983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.772098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.772256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.772495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.772575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.772782 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.772916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.773069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.773338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.773610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.773881 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.774154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.774420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.803451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.803690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.805273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.805368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.805508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.805629 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.810798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.810913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.811080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.811225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.812030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.812176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.819353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.819448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.819632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.819712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.824122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.824259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.824528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.824608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.824811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.824910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.825119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.825200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.825397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.825489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.825672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.825752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.825938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.826015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.826203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.826283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.848388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.848663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.849541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.849823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.853287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.854532 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.856939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.857236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.858266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.858565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.862349 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.863624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.866074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.867251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.869864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.870166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.870463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.870747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.871040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.871205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.871327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.871477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.871618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.871902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.872193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.872474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.891344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.891712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.891981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.892245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.892632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.892937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.893207 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.893472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.900226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.901513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.901976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.902253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.903225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.903501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.905940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.906278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.906565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.906850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.907127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.907223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.907506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.907788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.908072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.908347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.920988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.921343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.921631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.921915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.922198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.922287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.922477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.922557 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.922764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.922844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.923043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.923319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.923599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.923887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.941504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.942749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.944775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.946042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.948111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.949398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.949767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.951023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.951394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.951681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.952640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.953342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.953625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.953903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.960102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.961380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.961823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.963016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.965094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.965387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.965675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.965989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.966487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.966787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.967081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.967409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.975515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.975804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.976090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.976649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.976984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.977393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.984091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.984369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.984649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.984960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.985236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.985508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.985746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:14:50.985840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.986026 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.986106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.986305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.986384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.986577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.986658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.986845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.986934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.987119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.987249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.987382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.987466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.987647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.987729 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.987927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.988199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.988469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.988729 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.989027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:50.989309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:14:51.027073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55e50e50f9c0 pid: 80421 (default app) May 23 15:14:51.029180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55e50e50f9c0 pid: 80422 (default app) May 23 15:14:51.030966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bf585-39aa-4aa6-aae8-cf22e683f17a None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:14:51.031257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bf585-39aa-4aa6-aae8-cf22e683f17a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:14:51.031538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bf585-39aa-4aa6-aae8-cf22e683f17a None None] PATH_INFO: `/v3` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:14:51.035790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Mon May 23 15:14:51 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:14:59.127396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb481a0-c1ec-4bf1-813c-a9caefeaff49 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:14:59.127955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb481a0-c1ec-4bf1-813c-a9caefeaff49 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:14:59.127955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb481a0-c1ec-4bf1-813c-a9caefeaff49 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:14:59.132382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:14:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:14:59.137550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:14:59.138617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:14:59.138749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:14:59.214115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: INFO dbcounter [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] Registered counter for database keystone May 23 15:14:59.236208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG oslo_db.sqlalchemy.engines [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80421) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:14:59.242396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writer thread running {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:14:59.427179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG passlib.handlers.bcrypt [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80421) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 15:14:59.427514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG passlib.handlers.bcrypt [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80421) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 15:14:59.487845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:14:59.489491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:14:59.613439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:14:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 23 15:14:59.750799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb481a0-c1ec-4bf1-813c-a9caefeaff49 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:14:59.819991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: INFO dbcounter [None req-7eb481a0-c1ec-4bf1-813c-a9caefeaff49 None None] Registered counter for database keystone May 23 15:14:59.839712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG oslo_db.sqlalchemy.engines [None req-7eb481a0-c1ec-4bf1-813c-a9caefeaff49 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80422) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:14:59.845993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writer thread running {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:14:59.966923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb481a0-c1ec-4bf1-813c-a9caefeaff49 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:14:59.971246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e0332-c50c-477e-9e87-7e2204ec9737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:14:59.971918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e0332-c50c-477e-9e87-7e2204ec9737 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:14:59.972209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e0332-c50c-477e-9e87-7e2204ec9737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:14:59.972489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e0332-c50c-477e-9e87-7e2204ec9737 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:14:59.980635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e0332-c50c-477e-9e87-7e2204ec9737 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:00.032200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b90e0332-c50c-477e-9e87-7e2204ec9737 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 15:15:00.032546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e0332-c50c-477e-9e87-7e2204ec9737 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:00.042085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Mon May 23 15:14:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:15:00.048412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:00.062947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cad2f3-e84c-482d-8df0-6857d646d6a7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:00.067344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521905d5-f6ec-42df-8469-8e8011581eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:00.068465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521905d5-f6ec-42df-8469-8e8011581eed admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:00.068750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521905d5-f6ec-42df-8469-8e8011581eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:00.069054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521905d5-f6ec-42df-8469-8e8011581eed admin admin] PATH_INFO: `/v3/projects/c586cdfef8dc4cea9cf389817d345b08` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:00.070551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521905d5-f6ec-42df-8469-8e8011581eed admin admin] RBAC: Authorizing `identity:get_project(project_id=c586cdfef8dc4cea9cf389817d345b08)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:00.124071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WARNING keystone.common.rbac_enforcer.enforcer [None req-521905d5-f6ec-42df-8469-8e8011581eed admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 15:15:00.124406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521905d5-f6ec-42df-8469-8e8011581eed admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:00.129051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Mon May 23 15:15:00 2022] GET /identity/v3/projects/c586cdfef8dc4cea9cf389817d345b08 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:02.341941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd8bb2-c64f-4316-941f-67801cccc790 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:02.342475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd8bb2-c64f-4316-941f-67801cccc790 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:02.342475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd8bb2-c64f-4316-941f-67801cccc790 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:02.346639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:15:02.352021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe637b2-bda1-44f4-87d3-f04d43e867e7 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:02.352382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe637b2-bda1-44f4-87d3-f04d43e867e7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:02.352569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe637b2-bda1-44f4-87d3-f04d43e867e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:02.372248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ebe637b2-bda1-44f4-87d3-f04d43e867e7 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:02.374140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ebe637b2-bda1-44f4-87d3-f04d43e867e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:02.381922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 15:15:02.403884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd8bb2-c64f-4316-941f-67801cccc790 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:02.415772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd8bb2-c64f-4316-941f-67801cccc790 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:02.420554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34630d-7880-4a09-80a1-d77f7491ae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:02.421217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34630d-7880-4a09-80a1-d77f7491ae93 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:02.421501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34630d-7880-4a09-80a1-d77f7491ae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:02.421775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34630d-7880-4a09-80a1-d77f7491ae93 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:02.429394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34630d-7880-4a09-80a1-d77f7491ae93 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:02.431171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34630d-7880-4a09-80a1-d77f7491ae93 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:02.440370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Mon May 23 15:15:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:15:02.445524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe637b2-bda1-44f4-87d3-f04d43e867e7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:02.454580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe637b2-bda1-44f4-87d3-f04d43e867e7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:02.458985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50303b4a-8902-4763-a9b3-ea378c985c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:02.459842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50303b4a-8902-4763-a9b3-ea378c985c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:02.460127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50303b4a-8902-4763-a9b3-ea378c985c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:02.460402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50303b4a-8902-4763-a9b3-ea378c985c84 admin admin] PATH_INFO: `/v3/users/61ee072739a04036a34cf038cd44e02b` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:02.461800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50303b4a-8902-4763-a9b3-ea378c985c84 admin admin] RBAC: Authorizing `identity:get_user(user_id=61ee072739a04036a34cf038cd44e02b)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:02.463637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50303b4a-8902-4763-a9b3-ea378c985c84 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:02.465257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Mon May 23 15:15:02 2022] GET /identity/v3/users/61ee072739a04036a34cf038cd44e02b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:04.728111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c2b3a-cad9-451b-a30b-ce47b0ff2e59 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:04.728627 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c2b3a-cad9-451b-a30b-ce47b0ff2e59 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:04.728719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c2b3a-cad9-451b-a30b-ce47b0ff2e59 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:04.730046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:04.735736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5336d0-f14b-48cd-8225-41e460213657 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:04.736138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5336d0-f14b-48cd-8225-41e460213657 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:04.736290 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5336d0-f14b-48cd-8225-41e460213657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:04.753135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-5d5336d0-f14b-48cd-8225-41e460213657 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:04.755242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5d5336d0-f14b-48cd-8225-41e460213657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:04.779045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71c8e7-efb5-4522-a893-6db1a50445ad None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:04.779330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71c8e7-efb5-4522-a893-6db1a50445ad None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:04.779611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71c8e7-efb5-4522-a893-6db1a50445ad None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:04.780992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:15:04.785950 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1403052-c353-4687-a58f-19e8b711e544 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:04.786239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1403052-c353-4687-a58f-19e8b711e544 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:04.786523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1403052-c353-4687-a58f-19e8b711e544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:04.826785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG passlib.handlers.bcrypt [None req-b1403052-c353-4687-a58f-19e8b711e544 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80422) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 15:15:04.827224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG passlib.handlers.bcrypt [None req-b1403052-c353-4687-a58f-19e8b711e544 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80422) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 15:15:04.835063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:04.859528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5336d0-f14b-48cd-8225-41e460213657 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:04.875080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b1403052-c353-4687-a58f-19e8b711e544 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:04.876683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b1403052-c353-4687-a58f-19e8b711e544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:04.882769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:04.964535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5336d0-f14b-48cd-8225-41e460213657 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:04.982766 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:04.999605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684ea7c-5373-45b1-981b-c1bd00fec9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:05.000400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684ea7c-5373-45b1-981b-c1bd00fec9db None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:05.000680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684ea7c-5373-45b1-981b-c1bd00fec9db None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:05.000990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684ea7c-5373-45b1-981b-c1bd00fec9db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:05.004616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1403052-c353-4687-a58f-19e8b711e544 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:05.012217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684ea7c-5373-45b1-981b-c1bd00fec9db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-a684ea7c-5373-45b1-981b-c1bd00fec9db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:05.023819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:05.025052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:05.026163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:15:05.031820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684ea7c-5373-45b1-981b-c1bd00fec9db None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:05.219203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1403052-c353-4687-a58f-19e8b711e544 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:05.221517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:05.228714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:05.251600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684ea7c-5373-45b1-981b-c1bd00fec9db None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:05.257761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fada4-2fc9-4bc5-b330-bed946dba757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:05.259301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fada4-2fc9-4bc5-b330-bed946dba757 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:05.259596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fada4-2fc9-4bc5-b330-bed946dba757 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:05.262277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fada4-2fc9-4bc5-b330-bed946dba757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:05.288412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ccb7a-0786-4b24-8318-0631c44451e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:05.289208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ccb7a-0786-4b24-8318-0631c44451e2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:05.289490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ccb7a-0786-4b24-8318-0631c44451e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:05.289764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ccb7a-0786-4b24-8318-0631c44451e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:05.290518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ccb7a-0786-4b24-8318-0631c44451e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:05.290822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ccb7a-0786-4b24-8318-0631c44451e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:05.292220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ccb7a-0786-4b24-8318-0631c44451e2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:05.304226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:05.309772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ccb7a-0786-4b24-8318-0631c44451e2 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:05.335742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fada4-2fc9-4bc5-b330-bed946dba757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:05.337293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fada4-2fc9-4bc5-b330-bed946dba757 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:05.408309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ccb7a-0786-4b24-8318-0631c44451e2 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:05.434153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 8/16] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:15:05.439385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fada4-2fc9-4bc5-b330-bed946dba757 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:05.441498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b7c3-bdd1-411a-8aff-6e86c99bc655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:05.442091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:05.442374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b7c3-bdd1-411a-8aff-6e86c99bc655 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:05.442658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b7c3-bdd1-411a-8aff-6e86c99bc655 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:05.442936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b7c3-bdd1-411a-8aff-6e86c99bc655 None admin] PATH_INFO: `/v3/users/61ee072739a04036a34cf038cd44e02b` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:05.444313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6b7c3-bdd1-411a-8aff-6e86c99bc655 None admin] RBAC: Authorizing `identity:get_user(user_id=61ee072739a04036a34cf038cd44e02b)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:05.445813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6b7c3-bdd1-411a-8aff-6e86c99bc655 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:05.447434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 9/17] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:05 2022] GET /identity/v3/users/61ee072739a04036a34cf038cd44e02b => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:05.453488 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b7c3-bdd1-411a-8aff-6e86c99bc655 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:05.468669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:05.540823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fada4-2fc9-4bc5-b330-bed946dba757 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:05.557066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b7c3-bdd1-411a-8aff-6e86c99bc655 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:05.577164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4ac4b-61cc-4083-871e-0b27f3186012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:05.577956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ac4b-61cc-4083-871e-0b27f3186012 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:05.578234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ac4b-61cc-4083-871e-0b27f3186012 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:05.578503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ac4b-61cc-4083-871e-0b27f3186012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:05.604088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4ac4b-61cc-4083-871e-0b27f3186012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:05.605680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d41ab1-f082-45e3-9638-9ee393a51bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:05.605902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4ac4b-61cc-4083-871e-0b27f3186012 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:05.606613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d41ab1-f082-45e3-9638-9ee393a51bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:05.606898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d41ab1-f082-45e3-9638-9ee393a51bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:05.607175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d41ab1-f082-45e3-9638-9ee393a51bd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:05.608591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d41ab1-f082-45e3-9638-9ee393a51bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:05.610132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d41ab1-f082-45e3-9638-9ee393a51bd2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:05.611731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 10/18] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:05.617316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d41ab1-f082-45e3-9638-9ee393a51bd2 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-2ff4ac4b-61cc-4083-871e-0b27f3186012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:15:05.627580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:05.628558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:05.629875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:15:05.629875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:15:05.629875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:15:05.629875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:05.629875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 9/19] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:15:05.634491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4ac4b-61cc-4083-871e-0b27f3186012 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:05.692486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:05.698719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:05.718613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d41ab1-f082-45e3-9638-9ee393a51bd2 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:05.738014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4ac4b-61cc-4083-871e-0b27f3186012 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:05.752286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d590e15-9188-4f7b-a674-2c67511a498e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:05.752963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590e15-9188-4f7b-a674-2c67511a498e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:05.753244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590e15-9188-4f7b-a674-2c67511a498e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:05.753515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590e15-9188-4f7b-a674-2c67511a498e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:05.754340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d590e15-9188-4f7b-a674-2c67511a498e None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.domain.id=default, user.id=61ee072739a04036a34cf038cd44e02b) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:05.754645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d590e15-9188-4f7b-a674-2c67511a498e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:05.756093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d590e15-9188-4f7b-a674-2c67511a498e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:05.763209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 11/20] 10.222.0.30 () {56 vars in 1322 bytes} [Mon May 23 15:15:05 2022] GET /identity/v3/role_assignments?user.id=61ee072739a04036a34cf038cd44e02b&scope.domain.id=default&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:05.771187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29111fc-9106-4382-b6d3-c190b4d045b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:05.771956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29111fc-9106-4382-b6d3-c190b4d045b7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:05.772244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29111fc-9106-4382-b6d3-c190b4d045b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:05.772522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29111fc-9106-4382-b6d3-c190b4d045b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:05.773301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29111fc-9106-4382-b6d3-c190b4d045b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:05.773611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29111fc-9106-4382-b6d3-c190b4d045b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:05.775069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29111fc-9106-4382-b6d3-c190b4d045b7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:05.795824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 10/21] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:08.094070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb8ab-8b8f-4a09-942c-73b4efb72c9d None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:08.095106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb8ab-8b8f-4a09-942c-73b4efb72c9d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:08.095590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb8ab-8b8f-4a09-942c-73b4efb72c9d None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:08.097196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 12/22] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:08.102704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f6898-8aa8-48ab-938d-e0d6a048e724 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:08.103513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f6898-8aa8-48ab-938d-e0d6a048e724 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:08.104004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f6898-8aa8-48ab-938d-e0d6a048e724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:08.127776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-8d5f6898-8aa8-48ab-938d-e0d6a048e724 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:08.129328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8d5f6898-8aa8-48ab-938d-e0d6a048e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:08.215394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 11/23] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:08.249894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb8ab-8b8f-4a09-942c-73b4efb72c9d None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:08.382365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb8ab-8b8f-4a09-942c-73b4efb72c9d None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:08.432975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2a798-690a-4866-b4e2-881eeb45ca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:08.434288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2a798-690a-4866-b4e2-881eeb45ca30 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:08.434766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2a798-690a-4866-b4e2-881eeb45ca30 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:08.435234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2a798-690a-4866-b4e2-881eeb45ca30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:08.458012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2a798-690a-4866-b4e2-881eeb45ca30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-81b2a798-690a-4866-b4e2-881eeb45ca30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:08.467299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:08.468196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:08.468968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:08.471466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 13/24] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:08.477466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f6898-8aa8-48ab-938d-e0d6a048e724 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:08.541962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:08.608932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f6898-8aa8-48ab-938d-e0d6a048e724 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:08.662208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ea6c9-cdf3-4be2-a62f-33c6f7d7cbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:08.662821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ea6c9-cdf3-4be2-a62f-33c6f7d7cbce None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:08.663108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ea6c9-cdf3-4be2-a62f-33c6f7d7cbce None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:08.663391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ea6c9-cdf3-4be2-a62f-33c6f7d7cbce None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:08.664160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ea6c9-cdf3-4be2-a62f-33c6f7d7cbce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:08.664471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ea6c9-cdf3-4be2-a62f-33c6f7d7cbce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:08.665901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ea6c9-cdf3-4be2-a62f-33c6f7d7cbce None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:08.686256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdaee9-f073-41ee-aea6-893cc4cfad95 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:08.686550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdaee9-f073-41ee-aea6-893cc4cfad95 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:08.686816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdaee9-f073-41ee-aea6-893cc4cfad95 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:08.689124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 14/25] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:08.689407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 12/26] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:08.693830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cdaee9-f073-41ee-aea6-893cc4cfad95 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:08.697993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2970f-4356-4815-b08c-b742f448fef5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:08.698282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2970f-4356-4815-b08c-b742f448fef5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:08.698561 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2970f-4356-4815-b08c-b742f448fef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:08.709612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:08.728245 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-c9c2970f-4356-4815-b08c-b742f448fef5 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:08.729895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-c9c2970f-4356-4815-b08c-b742f448fef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:08.810595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 13/27] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:08.822526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746244-16d6-4e2b-86e7-5adc85fee6e8 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:08.823014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746244-16d6-4e2b-86e7-5adc85fee6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:08.823436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746244-16d6-4e2b-86e7-5adc85fee6e8 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:08.825074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 14/28] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:08.830095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcba846-1002-4195-a271-3cb2a354d208 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:08.830638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcba846-1002-4195-a271-3cb2a354d208 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:08.831191 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcba846-1002-4195-a271-3cb2a354d208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:08.857167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4fcba846-1002-4195-a271-3cb2a354d208 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:08.858870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4fcba846-1002-4195-a271-3cb2a354d208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:08.879379 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cdaee9-f073-41ee-aea6-893cc4cfad95 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:08.914088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:08.948091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c07282-2dc3-42de-96b2-6ae572985840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:08.953254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c07282-2dc3-42de-96b2-6ae572985840 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:08.954077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c07282-2dc3-42de-96b2-6ae572985840 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:08.954672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c07282-2dc3-42de-96b2-6ae572985840 None admin] PATH_INFO: `/v3/users/61ee072739a04036a34cf038cd44e02b` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:08.959629 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c07282-2dc3-42de-96b2-6ae572985840 None admin] RBAC: Authorizing `identity:get_user(user_id=61ee072739a04036a34cf038cd44e02b)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:08.962262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c07282-2dc3-42de-96b2-6ae572985840 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:08.969598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 15/29] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:08 2022] GET /identity/v3/users/61ee072739a04036a34cf038cd44e02b => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:08.970638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c07282-2dc3-42de-96b2-6ae572985840 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:08.991092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 15/30] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:08.992229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcba846-1002-4195-a271-3cb2a354d208 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:09.077245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:09.096794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c07282-2dc3-42de-96b2-6ae572985840 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:09.110996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcba846-1002-4195-a271-3cb2a354d208 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:09.147856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8fb83-56b5-47e9-b44c-b1cabe3a9e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:09.149078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8fb83-56b5-47e9-b44c-b1cabe3a9e72 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:09.149562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8fb83-56b5-47e9-b44c-b1cabe3a9e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:09.150051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8fb83-56b5-47e9-b44c-b1cabe3a9e72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:09.152336 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8fb83-56b5-47e9-b44c-b1cabe3a9e72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:09.154099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8fb83-56b5-47e9-b44c-b1cabe3a9e72 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:09.156328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 16/31] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:09.157464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8fb83-56b5-47e9-b44c-b1cabe3a9e72 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:09.162496 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685724a-66da-475b-884c-681ab6f3a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:09.163008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685724a-66da-475b-884c-681ab6f3a3b8 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:09.163292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685724a-66da-475b-884c-681ab6f3a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:09.163590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685724a-66da-475b-884c-681ab6f3a3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:09.164370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685724a-66da-475b-884c-681ab6f3a3b8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:09.167562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685724a-66da-475b-884c-681ab6f3a3b8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:09.198356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:09.270590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8fb83-56b5-47e9-b44c-b1cabe3a9e72 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:09.306416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0079f-312c-420f-990d-bd774369a2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:09.307426 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0079f-312c-420f-990d-bd774369a2ca None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:09.307851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0079f-312c-420f-990d-bd774369a2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:09.308924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0079f-312c-420f-990d-bd774369a2ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:09.321771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0079f-312c-420f-990d-bd774369a2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:09.323424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0079f-312c-420f-990d-bd774369a2ca None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:09.329749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 16/32] 10.222.0.30 () {60 vars in 1131 bytes} [Mon May 23 15:15:08 2022] POST /identity/v3/roles => generated 225 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:09.330833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685724a-66da-475b-884c-681ab6f3a3b8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-92d0079f-312c-420f-990d-bd774369a2ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:15:09.356038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:09.356951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:09.357741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:15:09.357741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:15:09.357741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:15:09.357741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:09.357741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 17/33] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:09.358803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afb785-32ac-49c1-bba4-52d45a41ed8c None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:09.359985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afb785-32ac-49c1-bba4-52d45a41ed8c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:09.359985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afb785-32ac-49c1-bba4-52d45a41ed8c None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:09.369802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 18/34] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:09.370866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afb785-32ac-49c1-bba4-52d45a41ed8c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:09.489377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:09.507140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afb785-32ac-49c1-bba4-52d45a41ed8c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:09.548733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63491a4-de79-4017-a19c-21ca893f3e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:09.549439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63491a4-de79-4017-a19c-21ca893f3e0c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:09.549716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63491a4-de79-4017-a19c-21ca893f3e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:09.549986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63491a4-de79-4017-a19c-21ca893f3e0c None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:09.551251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63491a4-de79-4017-a19c-21ca893f3e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:09.551560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63491a4-de79-4017-a19c-21ca893f3e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:09.553290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63491a4-de79-4017-a19c-21ca893f3e0c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:09.568322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 19/35] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:09.569648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d579d21-d944-46d8-94f6-9eeff6b46c92 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:09.569933 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d579d21-d944-46d8-94f6-9eeff6b46c92 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:09.570205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d579d21-d944-46d8-94f6-9eeff6b46c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:09.590539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-5d579d21-d944-46d8-94f6-9eeff6b46c92 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:09.596903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5d579d21-d944-46d8-94f6-9eeff6b46c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:09.600916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685724a-66da-475b-884c-681ab6f3a3b8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:09.654403 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:09.663208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be81b2-dd8e-48e8-bda6-435ec88e602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:09.664115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be81b2-dd8e-48e8-bda6-435ec88e602b None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:09.664402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be81b2-dd8e-48e8-bda6-435ec88e602b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:09.664682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be81b2-dd8e-48e8-bda6-435ec88e602b None admin] PATH_INFO: `/v3/domains/default/users/61ee072739a04036a34cf038cd44e02b/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:09.667302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be81b2-dd8e-48e8-bda6-435ec88e602b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=61ee072739a04036a34cf038cd44e02b, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:09.670830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be81b2-dd8e-48e8-bda6-435ec88e602b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:09.696935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 17/36] 10.222.0.30 () {58 vars in 1361 bytes} [Mon May 23 15:15:09 2022] PUT /identity/v3/domains/default/users/61ee072739a04036a34cf038cd44e02b/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:09.696935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be81b2-dd8e-48e8-bda6-435ec88e602b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:09.738090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 20/37] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:09.782729 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d579d21-d944-46d8-94f6-9eeff6b46c92 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:09.897094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:09.926310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d579d21-d944-46d8-94f6-9eeff6b46c92 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:09.927197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be81b2-dd8e-48e8-bda6-435ec88e602b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:09.966265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a8b97-6230-40e4-91f6-3cf8ace658ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:09.967319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a8b97-6230-40e4-91f6-3cf8ace658ec None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:09.967747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a8b97-6230-40e4-91f6-3cf8ace658ec None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:09.968156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a8b97-6230-40e4-91f6-3cf8ace658ec None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:09.969074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a8b97-6230-40e4-91f6-3cf8ace658ec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:09.970719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a8b97-6230-40e4-91f6-3cf8ace658ec None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:10.011644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654348a5-5228-4c13-959c-5174f2de75fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:10.011644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654348a5-5228-4c13-959c-5174f2de75fa None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:10.011644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654348a5-5228-4c13-959c-5174f2de75fa None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:10.011644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654348a5-5228-4c13-959c-5174f2de75fa None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:10.013343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654348a5-5228-4c13-959c-5174f2de75fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:10.015229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654348a5-5228-4c13-959c-5174f2de75fa None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:10.063857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 18/38] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:15:09 2022] POST /identity/v3/projects => generated 301 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:10.094773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406a626-56bd-4d6e-a91b-0dfbe1e8106f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:10.095514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406a626-56bd-4d6e-a91b-0dfbe1e8106f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:10.095984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406a626-56bd-4d6e-a91b-0dfbe1e8106f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:10.097786 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 19/39] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:10.110900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f44f2-2697-4f4b-b8ce-7a79c1bfb7e8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:10.111505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f44f2-2697-4f4b-b8ce-7a79c1bfb7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:10.111960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f44f2-2697-4f4b-b8ce-7a79c1bfb7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:10.139171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 21/40] 10.222.0.30 () {60 vars in 1131 bytes} [Mon May 23 15:15:09 2022] POST /identity/v3/roles => generated 219 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:15:10.146218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-5a2f44f2-2697-4f4b-b8ce-7a79c1bfb7e8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:10.147929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5a2f44f2-2697-4f4b-b8ce-7a79c1bfb7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:10.206584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eda2f2-0efc-4256-ac51-e211e0f51ae3 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:10.206876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eda2f2-0efc-4256-ac51-e211e0f51ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:10.207302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eda2f2-0efc-4256-ac51-e211e0f51ae3 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:10.208772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 22/41] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:15:10.219919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d56481-b87a-4dab-b9ee-663cb1c73581 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:10.220204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d56481-b87a-4dab-b9ee-663cb1c73581 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:10.220478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d56481-b87a-4dab-b9ee-663cb1c73581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:10.236256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:10.262377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-c5d56481-b87a-4dab-b9ee-663cb1c73581 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:10.266504 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-c5d56481-b87a-4dab-b9ee-663cb1c73581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:10.288417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 20/42] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:10.313099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f44f2-2697-4f4b-b8ce-7a79c1bfb7e8 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:10.371199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 23/43] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:10.396301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d56481-b87a-4dab-b9ee-663cb1c73581 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:10.450302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f44f2-2697-4f4b-b8ce-7a79c1bfb7e8 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:10.486711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:10.490177 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c70d03-18fa-4afe-b6e9-034970ac293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:10.490873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c70d03-18fa-4afe-b6e9-034970ac293a None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:10.491164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c70d03-18fa-4afe-b6e9-034970ac293a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:10.491441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c70d03-18fa-4afe-b6e9-034970ac293a None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:10.492320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c70d03-18fa-4afe-b6e9-034970ac293a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:10.494454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c70d03-18fa-4afe-b6e9-034970ac293a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:10.512640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 21/44] 10.222.0.30 () {60 vars in 1131 bytes} [Mon May 23 15:15:10 2022] POST /identity/v3/roles => generated 223 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:15:10.529931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d56481-b87a-4dab-b9ee-663cb1c73581 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:10.569560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99e252-4a48-4007-ae94-b8e1672c2fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:10.570449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99e252-4a48-4007-ae94-b8e1672c2fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:10.570742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99e252-4a48-4007-ae94-b8e1672c2fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:10.571023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99e252-4a48-4007-ae94-b8e1672c2fe9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:10.572390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99e252-4a48-4007-ae94-b8e1672c2fe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:10.574250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99e252-4a48-4007-ae94-b8e1672c2fe9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:10.576258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 24/45] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:10.581856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c70d03-18fa-4afe-b6e9-034970ac293a None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:10.628132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:10.703333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c70d03-18fa-4afe-b6e9-034970ac293a None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:10.740255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb36eae-d141-435f-beab-e8c3e5b210cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:10.741064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb36eae-d141-435f-beab-e8c3e5b210cd None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:10.741353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb36eae-d141-435f-beab-e8c3e5b210cd None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:10.741632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb36eae-d141-435f-beab-e8c3e5b210cd None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:10.742340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb36eae-d141-435f-beab-e8c3e5b210cd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:10.743821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb36eae-d141-435f-beab-e8c3e5b210cd None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:10.769701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 22/46] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:15:10 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:12.203935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d5fd-989d-44f1-9a52-8aaa56effc28 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:12.204443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d5fd-989d-44f1-9a52-8aaa56effc28 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:12.204553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d5fd-989d-44f1-9a52-8aaa56effc28 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:12.205902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 25/47] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:12.211163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3477a-e7c2-4314-b5d2-e6fdc827c52f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:12.211569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3477a-e7c2-4314-b5d2-e6fdc827c52f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:12.211720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3477a-e7c2-4314-b5d2-e6fdc827c52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:12.228811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-26e3477a-e7c2-4314-b5d2-e6fdc827c52f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:12.230287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-26e3477a-e7c2-4314-b5d2-e6fdc827c52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:12.295794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 23/48] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:12.319151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70d5fd-989d-44f1-9a52-8aaa56effc28 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:12.419739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70d5fd-989d-44f1-9a52-8aaa56effc28 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:12.453577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc63866-aac4-42c9-ab5c-10b40ebd0fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:12.454419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc63866-aac4-42c9-ab5c-10b40ebd0fca None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:12.454661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc63866-aac4-42c9-ab5c-10b40ebd0fca None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:12.454941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc63866-aac4-42c9-ab5c-10b40ebd0fca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:12.458506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:12.466803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc63866-aac4-42c9-ab5c-10b40ebd0fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-4fc63866-aac4-42c9-ab5c-10b40ebd0fca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:12.475611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:12.476413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:12.477206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 26/49] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:12.482892 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3477a-e7c2-4314-b5d2-e6fdc827c52f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:12.500945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:12.583515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3477a-e7c2-4314-b5d2-e6fdc827c52f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:12.619104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8026872-fd1c-451e-98e2-1a0391fbc986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:12.619782 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8026872-fd1c-451e-98e2-1a0391fbc986 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:12.620617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8026872-fd1c-451e-98e2-1a0391fbc986 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:12.620995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8026872-fd1c-451e-98e2-1a0391fbc986 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:12.622009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8026872-fd1c-451e-98e2-1a0391fbc986 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:12.622319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8026872-fd1c-451e-98e2-1a0391fbc986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:12.623716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8026872-fd1c-451e-98e2-1a0391fbc986 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:12.631939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 24/50] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:12.638136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc63866-aac4-42c9-ab5c-10b40ebd0fca None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:12.737479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc63866-aac4-42c9-ab5c-10b40ebd0fca None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:12.770365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0156d-0a17-4ff5-a796-84e5ad8c5cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:12.771266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0156d-0a17-4ff5-a796-84e5ad8c5cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:12.771555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0156d-0a17-4ff5-a796-84e5ad8c5cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:12.771835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0156d-0a17-4ff5-a796-84e5ad8c5cf4 None admin] PATH_INFO: `/v3/users/61ee072739a04036a34cf038cd44e02b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:12.773414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0156d-0a17-4ff5-a796-84e5ad8c5cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=61ee072739a04036a34cf038cd44e02b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:12.774921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0156d-0a17-4ff5-a796-84e5ad8c5cf4 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:12.776752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 27/51] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:12 2022] GET /identity/v3/users/61ee072739a04036a34cf038cd44e02b => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:12.782542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8026872-fd1c-451e-98e2-1a0391fbc986 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:12.863298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:12.884711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8026872-fd1c-451e-98e2-1a0391fbc986 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:12.917891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca174be7-6350-40c5-b3cc-2f9adfc03e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:12.918677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca174be7-6350-40c5-b3cc-2f9adfc03e4e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:12.918961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca174be7-6350-40c5-b3cc-2f9adfc03e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:12.919242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca174be7-6350-40c5-b3cc-2f9adfc03e4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:12.920418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca174be7-6350-40c5-b3cc-2f9adfc03e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:12.921900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca174be7-6350-40c5-b3cc-2f9adfc03e4e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:12.923513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 25/52] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:12.929167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0156d-0a17-4ff5-a796-84e5ad8c5cf4 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:12.970527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c6f5-de62-413e-ada0-3973be5765bc None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:12.970815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c6f5-de62-413e-ada0-3973be5765bc None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:12.971096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c6f5-de62-413e-ada0-3973be5765bc None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:12.972471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 26/53] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:15:12.977444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbd206-7d9b-45a2-aa07-23c8c7fcba72 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:12.977734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbd206-7d9b-45a2-aa07-23c8c7fcba72 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:12.978039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbd206-7d9b-45a2-aa07-23c8c7fcba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:12.991012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:12.994862 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-87cbd206-7d9b-45a2-aa07-23c8c7fcba72 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:12.996429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-87cbd206-7d9b-45a2-aa07-23c8c7fcba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:13.029177 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0156d-0a17-4ff5-a796-84e5ad8c5cf4 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:13.063143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e936c49-6db2-422a-8b85-bc8b7d86e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:13.063834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e936c49-6db2-422a-8b85-bc8b7d86e814 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:13.064083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e936c49-6db2-422a-8b85-bc8b7d86e814 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:13.064361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e936c49-6db2-422a-8b85-bc8b7d86e814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:13.065222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e936c49-6db2-422a-8b85-bc8b7d86e814 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.domain.id=default, user.id=61ee072739a04036a34cf038cd44e02b) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:13.065534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e936c49-6db2-422a-8b85-bc8b7d86e814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:13.066971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e936c49-6db2-422a-8b85-bc8b7d86e814 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:13.067179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 27/54] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:13.076182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 28/55] 10.222.0.30 () {56 vars in 1322 bytes} [Mon May 23 15:15:12 2022] GET /identity/v3/role_assignments?user.id=61ee072739a04036a34cf038cd44e02b&scope.domain.id=default&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:13.092119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbd206-7d9b-45a2-aa07-23c8c7fcba72 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:13.177228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:13.203949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbd206-7d9b-45a2-aa07-23c8c7fcba72 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:13.240003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea6404-0b5a-457b-95ad-bc6e90c4d6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:13.240816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea6404-0b5a-457b-95ad-bc6e90c4d6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:13.241187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea6404-0b5a-457b-95ad-bc6e90c4d6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:13.241412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea6404-0b5a-457b-95ad-bc6e90c4d6d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:13.242628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea6404-0b5a-457b-95ad-bc6e90c4d6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:13.244112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea6404-0b5a-457b-95ad-bc6e90c4d6d7 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:13.246886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 28/56] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:13.253252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e936c49-6db2-422a-8b85-bc8b7d86e814 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:13.355487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e936c49-6db2-422a-8b85-bc8b7d86e814 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:13.380710 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:13.389294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af26f7e-cb8a-499d-bb87-4b74d37a2ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:13.389954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af26f7e-cb8a-499d-bb87-4b74d37a2ec6 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:13.390234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af26f7e-cb8a-499d-bb87-4b74d37a2ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:13.390510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af26f7e-cb8a-499d-bb87-4b74d37a2ec6 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:13.391226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af26f7e-cb8a-499d-bb87-4b74d37a2ec6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:13.392679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af26f7e-cb8a-499d-bb87-4b74d37a2ec6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:13.418701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 29/57] 10.222.0.30 () {60 vars in 1140 bytes} [Mon May 23 15:15:13 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:15:15.583224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3b001-a5ac-4d18-af67-6dcbeb5180b8 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:15.583768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3b001-a5ac-4d18-af67-6dcbeb5180b8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:15.583768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3b001-a5ac-4d18-af67-6dcbeb5180b8 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:15.585206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 29/58] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:15.591081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48428530-a8ad-4139-9da9-bc8356ee757d None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:15.591515 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48428530-a8ad-4139-9da9-bc8356ee757d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:15.591624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48428530-a8ad-4139-9da9-bc8356ee757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:15.608753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-48428530-a8ad-4139-9da9-bc8356ee757d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:15.609902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-48428530-a8ad-4139-9da9-bc8356ee757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:15.674156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 30/59] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:15.697564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3b001-a5ac-4d18-af67-6dcbeb5180b8 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:15.794843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3b001-a5ac-4d18-af67-6dcbeb5180b8 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:15.827591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e115639-6e52-412e-bfa2-7f120b404bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:15.828362 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e115639-6e52-412e-bfa2-7f120b404bbc None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:15.828639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e115639-6e52-412e-bfa2-7f120b404bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:15.828936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e115639-6e52-412e-bfa2-7f120b404bbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:15.830082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e115639-6e52-412e-bfa2-7f120b404bbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:15.831559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e115639-6e52-412e-bfa2-7f120b404bbc None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:15.833157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 30/60] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:15.838611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48428530-a8ad-4139-9da9-bc8356ee757d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:15.933520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48428530-a8ad-4139-9da9-bc8356ee757d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:15.943647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:15.981406 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81994dc-0cef-43a1-bb45-9aced391f4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:15.982082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81994dc-0cef-43a1-bb45-9aced391f4a2 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:15.982366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81994dc-0cef-43a1-bb45-9aced391f4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:15.982643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81994dc-0cef-43a1-bb45-9aced391f4a2 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:15.983352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81994dc-0cef-43a1-bb45-9aced391f4a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:15.984819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81994dc-0cef-43a1-bb45-9aced391f4a2 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:16.010903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 31/61] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:15:15 2022] POST /identity/v3/users => generated 275 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:19.545255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e0463-f4c9-48ff-8b26-2d5ba347b93c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.546271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e0463-f4c9-48ff-8b26-2d5ba347b93c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.546780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e0463-f4c9-48ff-8b26-2d5ba347b93c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.548370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 31/62] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:19.554152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a61860-fd31-44eb-ad64-46223fbe373f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.555075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a61860-fd31-44eb-ad64-46223fbe373f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.555555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a61860-fd31-44eb-ad64-46223fbe373f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.571381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f44e2-bddd-4bc4-a554-719ce12155af None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.571988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f44e2-bddd-4bc4-a554-719ce12155af None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.573042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f44e2-bddd-4bc4-a554-719ce12155af None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.575095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 32/63] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:19.576251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-79a61860-fd31-44eb-ad64-46223fbe373f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:19.577500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-79a61860-fd31-44eb-ad64-46223fbe373f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:19.580596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c57e78-e2ca-416c-9b7a-af930fdf9545 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.581145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c57e78-e2ca-416c-9b7a-af930fdf9545 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.581647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c57e78-e2ca-416c-9b7a-af930fdf9545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.613247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-70c57e78-e2ca-416c-9b7a-af930fdf9545 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:19.614721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-70c57e78-e2ca-416c-9b7a-af930fdf9545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:19.634705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:19.680496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 32/64] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:19.681877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99becf-f370-41b5-9fec-28220a2dc0d5 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.682338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99becf-f370-41b5-9fec-28220a2dc0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.682844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99becf-f370-41b5-9fec-28220a2dc0d5 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.684330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 33/65] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:19.703261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef66e5-d7e4-42bd-9c4b-fad48dc6cdd5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.703864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef66e5-d7e4-42bd-9c4b-fad48dc6cdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.704281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef66e5-d7e4-42bd-9c4b-fad48dc6cdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.723055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-66ef66e5-d7e4-42bd-9c4b-fad48dc6cdd5 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:19.724537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-66ef66e5-d7e4-42bd-9c4b-fad48dc6cdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:19.727066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 33/66] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:19.728992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c57e78-e2ca-416c-9b7a-af930fdf9545 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:19.804971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 34/67] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:19.804971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede89472-45c9-4a3f-b38d-d0a0d9ba2ecf None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.804971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede89472-45c9-4a3f-b38d-d0a0d9ba2ecf None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.804971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede89472-45c9-4a3f-b38d-d0a0d9ba2ecf None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.804971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 35/68] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:19.804971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede89472-45c9-4a3f-b38d-d0a0d9ba2ecf None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:19.836696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:19.851775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c57e78-e2ca-416c-9b7a-af930fdf9545 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:19.895858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f0041-79ce-471d-851f-47de540518b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:19.897236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f0041-79ce-471d-851f-47de540518b3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.897524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f0041-79ce-471d-851f-47de540518b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.897801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f0041-79ce-471d-851f-47de540518b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.911305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f0041-79ce-471d-851f-47de540518b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-573f0041-79ce-471d-851f-47de540518b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:19.919828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:19.920886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:19.921659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 34/69] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:19.922642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b215447-2c84-45b7-858c-6db0cc4e405d None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.922930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b215447-2c84-45b7-858c-6db0cc4e405d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.923210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b215447-2c84-45b7-858c-6db0cc4e405d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.929142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:19.929397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede89472-45c9-4a3f-b38d-d0a0d9ba2ecf None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:19.941586 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-7b215447-2c84-45b7-858c-6db0cc4e405d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:19.942779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7b215447-2c84-45b7-858c-6db0cc4e405d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:19.966839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee15bb-e014-4eb1-a090-28eab05fd2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:19.967650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee15bb-e014-4eb1-a090-28eab05fd2ca None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:19.968102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee15bb-e014-4eb1-a090-28eab05fd2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:19.968383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee15bb-e014-4eb1-a090-28eab05fd2ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:19.979688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee15bb-e014-4eb1-a090-28eab05fd2ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-fbee15bb-e014-4eb1-a090-28eab05fd2ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:19.987972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:19.988793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:19.989639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 36/70] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:15:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:19.991190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee15bb-e014-4eb1-a090-28eab05fd2ca None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.020364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 35/71] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:20.021556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b215447-2c84-45b7-858c-6db0cc4e405d None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.084166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:20.100304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee15bb-e014-4eb1-a090-28eab05fd2ca None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.132159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b215447-2c84-45b7-858c-6db0cc4e405d None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.134305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e747ca-dffa-48f0-b2b7-0406240e0954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.135751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e747ca-dffa-48f0-b2b7-0406240e0954 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.136032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e747ca-dffa-48f0-b2b7-0406240e0954 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.136310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e747ca-dffa-48f0-b2b7-0406240e0954 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.146269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e747ca-dffa-48f0-b2b7-0406240e0954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-62e747ca-dffa-48f0-b2b7-0406240e0954 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:20.154157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:20.154952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:20.155691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 37/72] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:20.156310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e747ca-dffa-48f0-b2b7-0406240e0954 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.166740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:20.172259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24536e0-50f5-43b2-9153-08af6d10e41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.172958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24536e0-50f5-43b2-9153-08af6d10e41d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.173247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24536e0-50f5-43b2-9153-08af6d10e41d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.173525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24536e0-50f5-43b2-9153-08af6d10e41d None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.174292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24536e0-50f5-43b2-9153-08af6d10e41d None admin] RBAC: Adding query filter params (name=member) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:20.174602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24536e0-50f5-43b2-9153-08af6d10e41d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:20.176040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24536e0-50f5-43b2-9153-08af6d10e41d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:20.184891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 36/73] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:20.185712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24536e0-50f5-43b2-9153-08af6d10e41d None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.261431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e747ca-dffa-48f0-b2b7-0406240e0954 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.285032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24536e0-50f5-43b2-9153-08af6d10e41d None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.294332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404bdf8-93ce-4230-b0ed-17df7c22f21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.295001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404bdf8-93ce-4230-b0ed-17df7c22f21d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.295288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404bdf8-93ce-4230-b0ed-17df7c22f21d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.295576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404bdf8-93ce-4230-b0ed-17df7c22f21d None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.296325 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404bdf8-93ce-4230-b0ed-17df7c22f21d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:20.296637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404bdf8-93ce-4230-b0ed-17df7c22f21d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:20.298068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404bdf8-93ce-4230-b0ed-17df7c22f21d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:20.305948 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 38/74] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:20.307716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ea53-b7a6-403f-8be1-046f1beab7a5 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.308020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ea53-b7a6-403f-8be1-046f1beab7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.308291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ea53-b7a6-403f-8be1-046f1beab7a5 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.309582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 39/75] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:20.310298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ea53-b7a6-403f-8be1-046f1beab7a5 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.320964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:20.324528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac728e4e-2bd5-4670-bfd6-fed4b9ef6037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.325366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac728e4e-2bd5-4670-bfd6-fed4b9ef6037 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.325646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac728e4e-2bd5-4670-bfd6-fed4b9ef6037 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.325941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac728e4e-2bd5-4670-bfd6-fed4b9ef6037 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.336065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac728e4e-2bd5-4670-bfd6-fed4b9ef6037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-ac728e4e-2bd5-4670-bfd6-fed4b9ef6037 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:20.343769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:20.344597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:20.345453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 37/76] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:20.346056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac728e4e-2bd5-4670-bfd6-fed4b9ef6037 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.403184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:20.414737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ea53-b7a6-403f-8be1-046f1beab7a5 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.449923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484425a-129b-4ff7-bd81-673a0a093de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.450114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac728e4e-2bd5-4670-bfd6-fed4b9ef6037 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.450593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484425a-129b-4ff7-bd81-673a0a093de6 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.450876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484425a-129b-4ff7-bd81-673a0a093de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.451153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484425a-129b-4ff7-bd81-673a0a093de6 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.451897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484425a-129b-4ff7-bd81-673a0a093de6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:20.452214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484425a-129b-4ff7-bd81-673a0a093de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:20.453622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484425a-129b-4ff7-bd81-673a0a093de6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:20.461931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 40/77] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:20.462723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484425a-129b-4ff7-bd81-673a0a093de6 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.484552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38be3e-ee67-4d9b-9a21-11c8a950c03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.485457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38be3e-ee67-4d9b-9a21-11c8a950c03e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.485744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38be3e-ee67-4d9b-9a21-11c8a950c03e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.486033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38be3e-ee67-4d9b-9a21-11c8a950c03e None admin] PATH_INFO: `/v3/users/61613a3a760a44eea15c4d454557a296` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.500402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38be3e-ee67-4d9b-9a21-11c8a950c03e None admin] RBAC: Authorizing `identity:get_user(user_id=61613a3a760a44eea15c4d454557a296)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:20.501947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38be3e-ee67-4d9b-9a21-11c8a950c03e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:20.503757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 38/78] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/users/61613a3a760a44eea15c4d454557a296 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:20.504786 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a07ae-361e-4459-a4c1-5d2463b1ff8c None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.505104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a07ae-361e-4459-a4c1-5d2463b1ff8c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.505394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a07ae-361e-4459-a4c1-5d2463b1ff8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.520979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-017a07ae-361e-4459-a4c1-5d2463b1ff8c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:20.522245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-017a07ae-361e-4459-a4c1-5d2463b1ff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:20.530400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:20.562023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484425a-129b-4ff7-bd81-673a0a093de6 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.593164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 39/79] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:20.593991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a07ae-361e-4459-a4c1-5d2463b1ff8c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.618591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b94f84-f807-4a20-bcdb-41d23043d566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.619736 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94f84-f807-4a20-bcdb-41d23043d566 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.620115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94f84-f807-4a20-bcdb-41d23043d566 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.620398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94f84-f807-4a20-bcdb-41d23043d566 None admin] PATH_INFO: `/v3/users/61613a3a760a44eea15c4d454557a296` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.621878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b94f84-f807-4a20-bcdb-41d23043d566 None admin] RBAC: Authorizing `identity:get_user(user_id=61613a3a760a44eea15c4d454557a296)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:20.623320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b94f84-f807-4a20-bcdb-41d23043d566 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:20.625136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 41/80] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/users/61613a3a760a44eea15c4d454557a296 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:20.625814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b94f84-f807-4a20-bcdb-41d23043d566 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.641233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:20.696813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a07ae-361e-4459-a4c1-5d2463b1ff8c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.724615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b94f84-f807-4a20-bcdb-41d23043d566 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.730544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce27213-a755-4923-b893-7a67702548c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.731683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce27213-a755-4923-b893-7a67702548c0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.732188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce27213-a755-4923-b893-7a67702548c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.732692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce27213-a755-4923-b893-7a67702548c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.734006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce27213-a755-4923-b893-7a67702548c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:20.734317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce27213-a755-4923-b893-7a67702548c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:20.735710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce27213-a755-4923-b893-7a67702548c0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:20.744985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 40/81] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:20.746401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce27213-a755-4923-b893-7a67702548c0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.763240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68542946-c573-4208-9a07-d3db5b23c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.764198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68542946-c573-4208-9a07-d3db5b23c3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.764492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68542946-c573-4208-9a07-d3db5b23c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.764772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68542946-c573-4208-9a07-d3db5b23c3d1 None admin] PATH_INFO: `/v3/users/61ee072739a04036a34cf038cd44e02b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.766004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68542946-c573-4208-9a07-d3db5b23c3d1 None admin] RBAC: Authorizing `identity:get_user(user_id=61ee072739a04036a34cf038cd44e02b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:20.767443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68542946-c573-4208-9a07-d3db5b23c3d1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:20.769119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 42/82] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/users/61ee072739a04036a34cf038cd44e02b => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:20.769881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68542946-c573-4208-9a07-d3db5b23c3d1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.778038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:20.853169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce27213-a755-4923-b893-7a67702548c0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.855936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:20.867843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68542946-c573-4208-9a07-d3db5b23c3d1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:20.888203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98af4c-83e5-49e8-b4c6-4fc90c91e994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.888852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98af4c-83e5-49e8-b4c6-4fc90c91e994 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.889169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98af4c-83e5-49e8-b4c6-4fc90c91e994 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.889448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98af4c-83e5-49e8-b4c6-4fc90c91e994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.903645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1be5b-5c08-4e05-9b76-0cca207d1da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:20.904434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1be5b-5c08-4e05-9b76-0cca207d1da7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:20.904718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1be5b-5c08-4e05-9b76-0cca207d1da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:20.905027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1be5b-5c08-4e05-9b76-0cca207d1da7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:20.906206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1be5b-5c08-4e05-9b76-0cca207d1da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:20.907638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1be5b-5c08-4e05-9b76-0cca207d1da7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:20.909348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 43/83] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:20.910121 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1be5b-5c08-4e05-9b76-0cca207d1da7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:20.959746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98af4c-83e5-49e8-b4c6-4fc90c91e994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:20.961273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98af4c-83e5-49e8-b4c6-4fc90c91e994 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:20.983054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:21.008451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1be5b-5c08-4e05-9b76-0cca207d1da7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:21.049039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7ec01-ffeb-4852-aaef-671fa5088a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:21.049697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7ec01-ffeb-4852-aaef-671fa5088a66 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:21.049992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7ec01-ffeb-4852-aaef-671fa5088a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:21.050273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7ec01-ffeb-4852-aaef-671fa5088a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:21.062614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 41/84] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:21.063467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98af4c-83e5-49e8-b4c6-4fc90c91e994 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:21.070091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:21.120577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7ec01-ffeb-4852-aaef-671fa5088a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:21.122290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7ec01-ffeb-4852-aaef-671fa5088a66 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:21.172433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98af4c-83e5-49e8-b4c6-4fc90c91e994 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:21.194087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:21.205889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ec30b-dff2-44eb-bb73-0ca17d4fc731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:21.206750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ec30b-dff2-44eb-bb73-0ca17d4fc731 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:21.207039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ec30b-dff2-44eb-bb73-0ca17d4fc731 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:21.207314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ec30b-dff2-44eb-bb73-0ca17d4fc731 None admin] PATH_INFO: `/v3/users/61613a3a760a44eea15c4d454557a296` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:21.208502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ec30b-dff2-44eb-bb73-0ca17d4fc731 None admin] RBAC: Authorizing `identity:get_user(user_id=61613a3a760a44eea15c4d454557a296)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:21.209999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ec30b-dff2-44eb-bb73-0ca17d4fc731 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:21.211621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 42/85] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:21 2022] GET /identity/v3/users/61613a3a760a44eea15c4d454557a296 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:21.212386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ec30b-dff2-44eb-bb73-0ca17d4fc731 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:21.220274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 44/86] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:21.221119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7ec01-ffeb-4852-aaef-671fa5088a66 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:21.284559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:21.314521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ec30b-dff2-44eb-bb73-0ca17d4fc731 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:21.319096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7ec01-ffeb-4852-aaef-671fa5088a66 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:21.348107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1cd9-85ba-4d27-a386-09f33389d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:21.348747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1cd9-85ba-4d27-a386-09f33389d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:21.349055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1cd9-85ba-4d27-a386-09f33389d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:21.349331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1cd9-85ba-4d27-a386-09f33389d2b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:21.352810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84a66e-af35-42bd-b028-e11432e33565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:21.354367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84a66e-af35-42bd-b028-e11432e33565 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:21.354652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84a66e-af35-42bd-b028-e11432e33565 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:21.354927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84a66e-af35-42bd-b028-e11432e33565 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:21.355645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84a66e-af35-42bd-b028-e11432e33565 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:21.357111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84a66e-af35-42bd-b028-e11432e33565 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:21.376516 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 45/87] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:15:21 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:21.377324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84a66e-af35-42bd-b028-e11432e33565 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:21.424784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:21.444637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe1cd9-85ba-4d27-a386-09f33389d2b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:21.446285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe1cd9-85ba-4d27-a386-09f33389d2b0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:21.507811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84a66e-af35-42bd-b028-e11432e33565 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:21.530195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:21.546940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918b2ce-0a02-4977-b83f-d987bcec9add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:21.548070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918b2ce-0a02-4977-b83f-d987bcec9add None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:21.548415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918b2ce-0a02-4977-b83f-d987bcec9add None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:21.548761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918b2ce-0a02-4977-b83f-d987bcec9add None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:21.550101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918b2ce-0a02-4977-b83f-d987bcec9add None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:21.551651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918b2ce-0a02-4977-b83f-d987bcec9add None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:21.558643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 46/88] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:21 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:21.558643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918b2ce-0a02-4977-b83f-d987bcec9add None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:21.573011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 43/89] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:21.573772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1cd9-85ba-4d27-a386-09f33389d2b0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:21.689208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:21.692850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918b2ce-0a02-4977-b83f-d987bcec9add None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:21.706562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1cd9-85ba-4d27-a386-09f33389d2b0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:21.734331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c820c3-e5b7-4eec-afcf-9ec79d9f30c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:21.735002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c820c3-e5b7-4eec-afcf-9ec79d9f30c9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:21.735518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c820c3-e5b7-4eec-afcf-9ec79d9f30c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:21.735632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c820c3-e5b7-4eec-afcf-9ec79d9f30c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:21.747736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228dc78-95e2-49f6-9246-6b3c32dbd389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:21.748596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228dc78-95e2-49f6-9246-6b3c32dbd389 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:21.748919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228dc78-95e2-49f6-9246-6b3c32dbd389 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:21.749424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228dc78-95e2-49f6-9246-6b3c32dbd389 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:21.750362 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228dc78-95e2-49f6-9246-6b3c32dbd389 None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:21.751811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228dc78-95e2-49f6-9246-6b3c32dbd389 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:21.753743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 44/90] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:21 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:21.754305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228dc78-95e2-49f6-9246-6b3c32dbd389 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:21.806108 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:21.830557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c820c3-e5b7-4eec-afcf-9ec79d9f30c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:21.832045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c820c3-e5b7-4eec-afcf-9ec79d9f30c9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:21.870760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228dc78-95e2-49f6-9246-6b3c32dbd389 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:21.909597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f902de5-765f-433a-9748-a6f83f6fe8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:21.910626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f902de5-765f-433a-9748-a6f83f6fe8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:21.911109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f902de5-765f-433a-9748-a6f83f6fe8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:21.911585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f902de5-765f-433a-9748-a6f83f6fe8f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:21.912625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f902de5-765f-433a-9748-a6f83f6fe8f8 None admin] RBAC: Adding query filter params (role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=84b308b5ae9a4e0482f37715f327cd20, user.id=61613a3a760a44eea15c4d454557a296) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:21.913179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f902de5-765f-433a-9748-a6f83f6fe8f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:21.914867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f902de5-765f-433a-9748-a6f83f6fe8f8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:21.922405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 45/91] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:21 2022] GET /identity/v3/role_assignments?user.id=61613a3a760a44eea15c4d454557a296&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=8de5789832e84f9388b1295d479fca6e => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:21.923477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f902de5-765f-433a-9748-a6f83f6fe8f8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:21.991630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:22.005697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 47/92] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:22.006986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c820c3-e5b7-4eec-afcf-9ec79d9f30c9 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:22.123380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f902de5-765f-433a-9748-a6f83f6fe8f8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:22.126335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:22.165503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a3f8-085a-45da-bd30-3df916fc024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:22.166562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a3f8-085a-45da-bd30-3df916fc024c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:22.166847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a3f8-085a-45da-bd30-3df916fc024c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:22.167126 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a3f8-085a-45da-bd30-3df916fc024c None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:22.167702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c820c3-e5b7-4eec-afcf-9ec79d9f30c9 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:22.168333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a3f8-085a-45da-bd30-3df916fc024c None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:22.169956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a3f8-085a-45da-bd30-3df916fc024c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:22.171785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 46/93] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:21 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:22.172572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a3f8-085a-45da-bd30-3df916fc024c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:22.205913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f970c6-0bd3-42a7-9ecd-03042e7968d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:22.206993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f970c6-0bd3-42a7-9ecd-03042e7968d7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:22.207489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f970c6-0bd3-42a7-9ecd-03042e7968d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:22.207961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f970c6-0bd3-42a7-9ecd-03042e7968d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:22.209037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f970c6-0bd3-42a7-9ecd-03042e7968d7 None admin] RBAC: Adding query filter params (role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=84b308b5ae9a4e0482f37715f327cd20, user.id=61613a3a760a44eea15c4d454557a296) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:22.209553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f970c6-0bd3-42a7-9ecd-03042e7968d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:22.211257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f970c6-0bd3-42a7-9ecd-03042e7968d7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:22.218810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 48/94] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:22 2022] GET /identity/v3/role_assignments?user.id=61613a3a760a44eea15c4d454557a296&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:22.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f970c6-0bd3-42a7-9ecd-03042e7968d7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:22.287289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a3f8-085a-45da-bd30-3df916fc024c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:22.322221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:22.335658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedb6dc-51a3-4206-acc5-2cfbbbbd71fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:22.336591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedb6dc-51a3-4206-acc5-2cfbbbbd71fc None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:22.336894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedb6dc-51a3-4206-acc5-2cfbbbbd71fc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:22.337174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedb6dc-51a3-4206-acc5-2cfbbbbd71fc None admin] PATH_INFO: `/v3/projects/82aa67c5532c41168f1d0faa3949e3c1` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:22.340599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedb6dc-51a3-4206-acc5-2cfbbbbd71fc None admin] RBAC: Authorizing `identity:get_project(project_id=82aa67c5532c41168f1d0faa3949e3c1)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:22.344325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedb6dc-51a3-4206-acc5-2cfbbbbd71fc None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:22.344325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 47/95] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:22 2022] GET /identity/v3/projects/82aa67c5532c41168f1d0faa3949e3c1 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:22.352420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedb6dc-51a3-4206-acc5-2cfbbbbd71fc None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:22.392570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f970c6-0bd3-42a7-9ecd-03042e7968d7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:22.436442 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:22.533463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedb6dc-51a3-4206-acc5-2cfbbbbd71fc None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:22.545673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1880d8f-76ce-437e-b7d0-8b27b0b35505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:22.546277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1880d8f-76ce-437e-b7d0-8b27b0b35505 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:22.546559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1880d8f-76ce-437e-b7d0-8b27b0b35505 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:22.546835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1880d8f-76ce-437e-b7d0-8b27b0b35505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:22.547675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1880d8f-76ce-437e-b7d0-8b27b0b35505 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=84b308b5ae9a4e0482f37715f327cd20, user.id=61ee072739a04036a34cf038cd44e02b) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:22.547984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1880d8f-76ce-437e-b7d0-8b27b0b35505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:22.549544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1880d8f-76ce-437e-b7d0-8b27b0b35505 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:22.570895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 49/96] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:22 2022] GET /identity/v3/role_assignments?user.id=61ee072739a04036a34cf038cd44e02b&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:22.574336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5b096-9a53-4135-9456-8e84d7cf21bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:22.575025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5b096-9a53-4135-9456-8e84d7cf21bc None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:22.575314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5b096-9a53-4135-9456-8e84d7cf21bc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:22.575599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5b096-9a53-4135-9456-8e84d7cf21bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:22.576464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5b096-9a53-4135-9456-8e84d7cf21bc None admin] RBAC: Adding query filter params (role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=82aa67c5532c41168f1d0faa3949e3c1, user.id=61613a3a760a44eea15c4d454557a296) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:22.576790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5b096-9a53-4135-9456-8e84d7cf21bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:22.578475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5b096-9a53-4135-9456-8e84d7cf21bc None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:22.585397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 48/97] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:22 2022] GET /identity/v3/role_assignments?user.id=61613a3a760a44eea15c4d454557a296&scope.project.id=82aa67c5532c41168f1d0faa3949e3c1&role.id=8de5789832e84f9388b1295d479fca6e => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:23.996910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07d3f1-5dfb-4c84-b5c0-dc5e998ff645 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:23.997625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07d3f1-5dfb-4c84-b5c0-dc5e998ff645 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:23.997625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07d3f1-5dfb-4c84-b5c0-dc5e998ff645 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:23.999035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 50/98] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:24.005933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35033f-eaab-4c19-b66c-0c54a60d41c6 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:24.006763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35033f-eaab-4c19-b66c-0c54a60d41c6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:24.007240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35033f-eaab-4c19-b66c-0c54a60d41c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:24.032444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4a35033f-eaab-4c19-b66c-0c54a60d41c6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:24.034018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4a35033f-eaab-4c19-b66c-0c54a60d41c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:24.123264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 49/99] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:24.159114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07d3f1-5dfb-4c84-b5c0-dc5e998ff645 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:24.300934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07d3f1-5dfb-4c84-b5c0-dc5e998ff645 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:24.339742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:24.348696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23371185-4dac-4c82-ae7e-4e3f23c9f34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:24.349522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23371185-4dac-4c82-ae7e-4e3f23c9f34d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:24.349806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23371185-4dac-4c82-ae7e-4e3f23c9f34d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:24.350079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23371185-4dac-4c82-ae7e-4e3f23c9f34d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:24.370221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23371185-4dac-4c82-ae7e-4e3f23c9f34d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-23371185-4dac-4c82-ae7e-4e3f23c9f34d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:24.378313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:24.379277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:24.380091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 51/100] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:24.402020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35033f-eaab-4c19-b66c-0c54a60d41c6 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:24.465583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:24.545354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35033f-eaab-4c19-b66c-0c54a60d41c6 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:24.608939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1b9b7-7d2f-4e21-b0d7-35daef791cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:24.609988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1b9b7-7d2f-4e21-b0d7-35daef791cda None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:24.609988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1b9b7-7d2f-4e21-b0d7-35daef791cda None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:24.610138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1b9b7-7d2f-4e21-b0d7-35daef791cda None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:24.610860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1b9b7-7d2f-4e21-b0d7-35daef791cda None admin] RBAC: Adding query filter params (name=reader) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:24.611171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1b9b7-7d2f-4e21-b0d7-35daef791cda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:24.612574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1b9b7-7d2f-4e21-b0d7-35daef791cda None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:24.632768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 50/101] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:24.642024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23371185-4dac-4c82-ae7e-4e3f23c9f34d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:24.817550 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23371185-4dac-4c82-ae7e-4e3f23c9f34d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:24.873993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8684ef-54ff-4647-ac1f-fa3db9deaf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:24.874804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8684ef-54ff-4647-ac1f-fa3db9deaf7c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:24.875088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8684ef-54ff-4647-ac1f-fa3db9deaf7c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:24.875363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8684ef-54ff-4647-ac1f-fa3db9deaf7c None admin] PATH_INFO: `/v3/users/7d91e07c08224d54af44714a692b4973` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:24.901253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8684ef-54ff-4647-ac1f-fa3db9deaf7c None admin] RBAC: Authorizing `identity:get_user(user_id=7d91e07c08224d54af44714a692b4973)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:24.902737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8684ef-54ff-4647-ac1f-fa3db9deaf7c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:24.904792 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 52/102] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:24 2022] GET /identity/v3/users/7d91e07c08224d54af44714a692b4973 => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:24.910399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1b9b7-7d2f-4e21-b0d7-35daef791cda None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:25.037258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:25.047359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1b9b7-7d2f-4e21-b0d7-35daef791cda None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:25.108228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1347002-aa20-4cc4-b4b3-b9cfae545e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:25.108228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1347002-aa20-4cc4-b4b3-b9cfae545e5f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.109091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1347002-aa20-4cc4-b4b3-b9cfae545e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.109091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1347002-aa20-4cc4-b4b3-b9cfae545e5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.156904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7867b-cc52-4075-884a-fa5067ab38dc None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.157397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7867b-cc52-4075-884a-fa5067ab38dc None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.157397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7867b-cc52-4075-884a-fa5067ab38dc None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.158798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 53/103] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:15:25.172840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e247d-cb06-4a04-bc8e-0af0709b82fb None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.173187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e247d-cb06-4a04-bc8e-0af0709b82fb None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.173450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e247d-cb06-4a04-bc8e-0af0709b82fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.201865 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-cf4e247d-cb06-4a04-bc8e-0af0709b82fb None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:25.203073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-cf4e247d-cb06-4a04-bc8e-0af0709b82fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:25.256505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1347002-aa20-4cc4-b4b3-b9cfae545e5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:25.258742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1347002-aa20-4cc4-b4b3-b9cfae545e5f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:25.322295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:25.386728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 54/104] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 23 15:15:25.406643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586049d9-1ae1-43f0-b129-03c52a1d2181 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.406643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586049d9-1ae1-43f0-b129-03c52a1d2181 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.406643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586049d9-1ae1-43f0-b129-03c52a1d2181 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.406643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 55/105] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:15:25.412021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68156830-f36f-4344-b172-3cd895d25c01 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.412251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68156830-f36f-4344-b172-3cd895d25c01 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.412536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68156830-f36f-4344-b172-3cd895d25c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.446445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-68156830-f36f-4344-b172-3cd895d25c01 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:25.449525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-68156830-f36f-4344-b172-3cd895d25c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:25.457395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:25.476650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 51/106] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:25.477895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1347002-aa20-4cc4-b4b3-b9cfae545e5f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:25.605720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 56/107] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:25.608695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1347002-aa20-4cc4-b4b3-b9cfae545e5f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:25.621278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68156830-f36f-4344-b172-3cd895d25c01 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:25.667221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b706a07-6c27-4a98-8673-926cb2415618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:25.668521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b706a07-6c27-4a98-8673-926cb2415618 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.669665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b706a07-6c27-4a98-8673-926cb2415618 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.670185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b706a07-6c27-4a98-8673-926cb2415618 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.672624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b706a07-6c27-4a98-8673-926cb2415618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:25.675387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b706a07-6c27-4a98-8673-926cb2415618 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:25.678239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 52/108] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:25.680350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba9cd5-8156-4be5-9fac-fa7aafccc9d2 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.680854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba9cd5-8156-4be5-9fac-fa7aafccc9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.681948 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba9cd5-8156-4be5-9fac-fa7aafccc9d2 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.688522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 53/109] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:25.688522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba9cd5-8156-4be5-9fac-fa7aafccc9d2 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:25.755361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68156830-f36f-4344-b172-3cd895d25c01 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:25.758623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:25.762818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:25.794668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a443b-d191-4f7b-bcef-e97a43012ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:25.795547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a443b-d191-4f7b-bcef-e97a43012ead None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.795834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a443b-d191-4f7b-bcef-e97a43012ead None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.796113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a443b-d191-4f7b-bcef-e97a43012ead None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.797461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a443b-d191-4f7b-bcef-e97a43012ead None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:25.799085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a443b-d191-4f7b-bcef-e97a43012ead None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:25.804957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 57/110] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:25 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:25.807847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a443b-d191-4f7b-bcef-e97a43012ead None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:25.820182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba9cd5-8156-4be5-9fac-fa7aafccc9d2 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:25.870146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e29e8d-5340-4cf6-8d8a-f78dcd583b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:25.871670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e29e8d-5340-4cf6-8d8a-f78dcd583b17 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.871981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e29e8d-5340-4cf6-8d8a-f78dcd583b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.872649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e29e8d-5340-4cf6-8d8a-f78dcd583b17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.886391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e29e8d-5340-4cf6-8d8a-f78dcd583b17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-81e29e8d-5340-4cf6-8d8a-f78dcd583b17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:25.897150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:25.897955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:25.898694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:25.899507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 54/111] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:25.901658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc7a21-9c43-432f-8e3e-6199229cedc0 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.901938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc7a21-9c43-432f-8e3e-6199229cedc0 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.902229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc7a21-9c43-432f-8e3e-6199229cedc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.919658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-3ffc7a21-9c43-432f-8e3e-6199229cedc0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:25.920755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3ffc7a21-9c43-432f-8e3e-6199229cedc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:25.937643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a443b-d191-4f7b-bcef-e97a43012ead None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:25.974075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007da5e-7ce7-4f2e-b1f6-1031d030d9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:25.975105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007da5e-7ce7-4f2e-b1f6-1031d030d9e3 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:25.975600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007da5e-7ce7-4f2e-b1f6-1031d030d9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:25.976088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007da5e-7ce7-4f2e-b1f6-1031d030d9e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:25.976897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007da5e-7ce7-4f2e-b1f6-1031d030d9e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:25.978937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007da5e-7ce7-4f2e-b1f6-1031d030d9e3 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:25.991241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 55/112] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:26.001075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc7a21-9c43-432f-8e3e-6199229cedc0 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:26.015246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 58/113] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:15:25 2022] POST /identity/v3/projects => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:26.016769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f990b5-1069-4382-893f-ed85249eae5b None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.017100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f990b5-1069-4382-893f-ed85249eae5b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.017405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f990b5-1069-4382-893f-ed85249eae5b None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.019009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 59/114] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:26.019732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f990b5-1069-4382-893f-ed85249eae5b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:26.049843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:26.068057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:26.124989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc7a21-9c43-432f-8e3e-6199229cedc0 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:26.138623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f990b5-1069-4382-893f-ed85249eae5b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:26.163994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d3b5b-1a7d-43c1-965a-22b51a25efde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:26.165058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d3b5b-1a7d-43c1-965a-22b51a25efde None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.165352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d3b5b-1a7d-43c1-965a-22b51a25efde None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.166193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d3b5b-1a7d-43c1-965a-22b51a25efde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.167633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d3b5b-1a7d-43c1-965a-22b51a25efde None admin] RBAC: Adding query filter params (role.id=2acd8a15a63a4261b1ae12f09d28722f, scope.project.id=84b308b5ae9a4e0482f37715f327cd20, user.id=7d91e07c08224d54af44714a692b4973) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:26.168124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d3b5b-1a7d-43c1-965a-22b51a25efde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:26.170430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d3b5b-1a7d-43c1-965a-22b51a25efde None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:26.176633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7035b33-5327-4645-9e0f-93031d1847b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:26.177382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7035b33-5327-4645-9e0f-93031d1847b9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.185324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7035b33-5327-4645-9e0f-93031d1847b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.185536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 56/115] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:25 2022] GET /identity/v3/role_assignments?user.id=7d91e07c08224d54af44714a692b4973&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=2acd8a15a63a4261b1ae12f09d28722f => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:26.185615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7035b33-5327-4645-9e0f-93031d1847b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.185615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7035b33-5327-4645-9e0f-93031d1847b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:26.185820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae11796-8143-4462-b9d6-6631709f89a0 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.185897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7035b33-5327-4645-9e0f-93031d1847b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:26.185963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae11796-8143-4462-b9d6-6631709f89a0 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.185963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae11796-8143-4462-b9d6-6631709f89a0 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.186073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7035b33-5327-4645-9e0f-93031d1847b9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:26.186144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 57/116] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:26.186144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae11796-8143-4462-b9d6-6631709f89a0 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:26.191433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 60/117] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:26.192565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb76016-2e26-45c8-862e-5dc42e7f01bb None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.192860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb76016-2e26-45c8-862e-5dc42e7f01bb None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.193192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb76016-2e26-45c8-862e-5dc42e7f01bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.215726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-dfb76016-2e26-45c8-862e-5dc42e7f01bb None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:26.219649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-dfb76016-2e26-45c8-862e-5dc42e7f01bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:26.313644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 61/118] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:26.320565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732d23e-5462-4a65-83e2-3c3de3885f18 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.320565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732d23e-5462-4a65-83e2-3c3de3885f18 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.320565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732d23e-5462-4a65-83e2-3c3de3885f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.343964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae11796-8143-4462-b9d6-6631709f89a0 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:26.350346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0732d23e-5462-4a65-83e2-3c3de3885f18 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:26.351920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0732d23e-5462-4a65-83e2-3c3de3885f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:26.368668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:26.370444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:26.395507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32008f-6e86-4dc3-94c0-dda96ce83c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:26.396427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32008f-6e86-4dc3-94c0-dda96ce83c8d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.396706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32008f-6e86-4dc3-94c0-dda96ce83c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.397001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32008f-6e86-4dc3-94c0-dda96ce83c8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.407929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32008f-6e86-4dc3-94c0-dda96ce83c8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-9e32008f-6e86-4dc3-94c0-dda96ce83c8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:26.417095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:26.418000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:26.418761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 58/119] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:15:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:26.419375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32008f-6e86-4dc3-94c0-dda96ce83c8d None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:26.447784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 62/120] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:26.448972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732d23e-5462-4a65-83e2-3c3de3885f18 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:26.541873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32008f-6e86-4dc3-94c0-dda96ce83c8d None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:26.580881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732d23e-5462-4a65-83e2-3c3de3885f18 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:26.584167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f3a52-a237-418b-a19c-59dccf769fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:26.591562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f3a52-a237-418b-a19c-59dccf769fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.592519 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f3a52-a237-418b-a19c-59dccf769fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.592971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f3a52-a237-418b-a19c-59dccf769fa3 None admin] PATH_INFO: `/v3/users/61613a3a760a44eea15c4d454557a296` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.595180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f3a52-a237-418b-a19c-59dccf769fa3 None admin] RBAC: Authorizing `identity:get_user(user_id=61613a3a760a44eea15c4d454557a296)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:26.597555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f3a52-a237-418b-a19c-59dccf769fa3 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:26.599694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 59/121] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:26 2022] GET /identity/v3/users/61613a3a760a44eea15c4d454557a296 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:26.600986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f3a52-a237-418b-a19c-59dccf769fa3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:26.625492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e2abc-75c5-4f19-b9dc-fcfac029348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:26.626831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e2abc-75c5-4f19-b9dc-fcfac029348a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.627722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e2abc-75c5-4f19-b9dc-fcfac029348a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.628153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e2abc-75c5-4f19-b9dc-fcfac029348a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.643770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e2abc-75c5-4f19-b9dc-fcfac029348a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-e46e2abc-75c5-4f19-b9dc-fcfac029348a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:26.655316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:26.656280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:26.657222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:26.661726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:26.662150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 63/122] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:26.668619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e2abc-75c5-4f19-b9dc-fcfac029348a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:26.682947 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:26.740843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f3a52-a237-418b-a19c-59dccf769fa3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:26.780125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c6ad2-cb8e-4d32-ad5e-8db93018726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:26.781663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c6ad2-cb8e-4d32-ad5e-8db93018726c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.782219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c6ad2-cb8e-4d32-ad5e-8db93018726c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.782646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c6ad2-cb8e-4d32-ad5e-8db93018726c None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.784189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c6ad2-cb8e-4d32-ad5e-8db93018726c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:26.784929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c6ad2-cb8e-4d32-ad5e-8db93018726c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:26.787365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c6ad2-cb8e-4d32-ad5e-8db93018726c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:26.789750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e2abc-75c5-4f19-b9dc-fcfac029348a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:26.801338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 60/123] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:15:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:26.802858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c6ad2-cb8e-4d32-ad5e-8db93018726c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:26.828859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769b83d-42f1-4e55-831c-7e8f0638efa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:26.829782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769b83d-42f1-4e55-831c-7e8f0638efa9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.830078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769b83d-42f1-4e55-831c-7e8f0638efa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.830404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769b83d-42f1-4e55-831c-7e8f0638efa9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.844379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769b83d-42f1-4e55-831c-7e8f0638efa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-2769b83d-42f1-4e55-831c-7e8f0638efa9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:26.852724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:26.854571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:26.855360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 64/124] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:26.855980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769b83d-42f1-4e55-831c-7e8f0638efa9 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:26.932418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:26.947545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c6ad2-cb8e-4d32-ad5e-8db93018726c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:26.960492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:26.976326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769b83d-42f1-4e55-831c-7e8f0638efa9 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:26.991143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079fbb-59d8-481e-8f64-bbfbbd02af65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:26.991964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079fbb-59d8-481e-8f64-bbfbbd02af65 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:26.992227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079fbb-59d8-481e-8f64-bbfbbd02af65 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:26.992523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079fbb-59d8-481e-8f64-bbfbbd02af65 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:26.993903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78079fbb-59d8-481e-8f64-bbfbbd02af65 None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:26.995374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78079fbb-59d8-481e-8f64-bbfbbd02af65 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:26.997386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 61/125] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:26 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:26.998253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079fbb-59d8-481e-8f64-bbfbbd02af65 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.023194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a017e-c392-4d6b-a6b1-9de2026dfcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:27.024842 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a017e-c392-4d6b-a6b1-9de2026dfcea None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:27.025983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a017e-c392-4d6b-a6b1-9de2026dfcea None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:27.026512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a017e-c392-4d6b-a6b1-9de2026dfcea None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:27.027938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a017e-c392-4d6b-a6b1-9de2026dfcea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:27.028697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a017e-c392-4d6b-a6b1-9de2026dfcea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:27.030764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a017e-c392-4d6b-a6b1-9de2026dfcea None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:27.041750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 65/126] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:27.046856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a017e-c392-4d6b-a6b1-9de2026dfcea None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.131525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079fbb-59d8-481e-8f64-bbfbbd02af65 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:27.158393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a017e-c392-4d6b-a6b1-9de2026dfcea None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:27.184160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbea9a-6337-462f-b165-108ac7136de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:27.185636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbea9a-6337-462f-b165-108ac7136de2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:27.186120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:27.188075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbea9a-6337-462f-b165-108ac7136de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:27.188608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbea9a-6337-462f-b165-108ac7136de2 None admin] PATH_INFO: `/v3/users/61613a3a760a44eea15c4d454557a296` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:27.190357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbea9a-6337-462f-b165-108ac7136de2 None admin] RBAC: Authorizing `identity:get_user(user_id=61613a3a760a44eea15c4d454557a296)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:27.193380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbea9a-6337-462f-b165-108ac7136de2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:27.195042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 62/127] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:26 2022] GET /identity/v3/users/61613a3a760a44eea15c4d454557a296 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:27.195800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbea9a-6337-462f-b165-108ac7136de2 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.197669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed25b96-2c63-4a1d-9af2-b9f681e01eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:27.198597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed25b96-2c63-4a1d-9af2-b9f681e01eee None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:27.199079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed25b96-2c63-4a1d-9af2-b9f681e01eee None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:27.199567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed25b96-2c63-4a1d-9af2-b9f681e01eee None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:27.200597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed25b96-2c63-4a1d-9af2-b9f681e01eee None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:27.201139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed25b96-2c63-4a1d-9af2-b9f681e01eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:27.202698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed25b96-2c63-4a1d-9af2-b9f681e01eee None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:27.211618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 66/128] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:27.213448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed25b96-2c63-4a1d-9af2-b9f681e01eee None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.258103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:27.327834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbea9a-6337-462f-b165-108ac7136de2 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:27.340693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed25b96-2c63-4a1d-9af2-b9f681e01eee None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:27.367510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d80300-1be3-4a41-8ac1-1ef318359ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:27.369116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80300-1be3-4a41-8ac1-1ef318359ebe None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:27.369633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80300-1be3-4a41-8ac1-1ef318359ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:27.370116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80300-1be3-4a41-8ac1-1ef318359ebe None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20/users/61613a3a760a44eea15c4d454557a296/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:27.372582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d80300-1be3-4a41-8ac1-1ef318359ebe None admin] RBAC: Authorizing `identity:create_grant(project_id=84b308b5ae9a4e0482f37715f327cd20, user_id=61613a3a760a44eea15c4d454557a296, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:27.374306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d80300-1be3-4a41-8ac1-1ef318359ebe None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:27.387330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61136645-f399-4e87-ae7a-402469bb400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:27.388225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61136645-f399-4e87-ae7a-402469bb400a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:27.388513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61136645-f399-4e87-ae7a-402469bb400a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:27.388795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61136645-f399-4e87-ae7a-402469bb400a None admin] PATH_INFO: `/v3/users/61ee072739a04036a34cf038cd44e02b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:27.390039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61136645-f399-4e87-ae7a-402469bb400a None admin] RBAC: Authorizing `identity:get_user(user_id=61ee072739a04036a34cf038cd44e02b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:27.390997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 63/129] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:15:27 2022] PUT /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20/users/61613a3a760a44eea15c4d454557a296/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:27.391501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61136645-f399-4e87-ae7a-402469bb400a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:27.392430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d80300-1be3-4a41-8ac1-1ef318359ebe None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.396964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 67/130] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:27 2022] GET /identity/v3/users/61ee072739a04036a34cf038cd44e02b => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:27.400354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61136645-f399-4e87-ae7a-402469bb400a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.529549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:27.581342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d80300-1be3-4a41-8ac1-1ef318359ebe None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:27.583829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:27.610428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61136645-f399-4e87-ae7a-402469bb400a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:27.630727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba535-d621-47e9-ae2e-176ded341fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:27.631891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba535-d621-47e9-ae2e-176ded341fdf None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:27.633117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba535-d621-47e9-ae2e-176ded341fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:27.633563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba535-d621-47e9-ae2e-176ded341fdf None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:27.636664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ba535-d621-47e9-ae2e-176ded341fdf None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:27.638386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ba535-d621-47e9-ae2e-176ded341fdf None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:27.642701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 64/131] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:27 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:27.643883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba535-d621-47e9-ae2e-176ded341fdf None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.677697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc1dee-32f3-4403-b017-785262563b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:27.678840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc1dee-32f3-4403-b017-785262563b0b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:27.679339 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc1dee-32f3-4403-b017-785262563b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:27.679600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc1dee-32f3-4403-b017-785262563b0b None admin] PATH_INFO: `/v3/users/61613a3a760a44eea15c4d454557a296` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:27.683758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc1dee-32f3-4403-b017-785262563b0b None admin] RBAC: Authorizing `identity:get_user(user_id=61613a3a760a44eea15c4d454557a296)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:27.685413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc1dee-32f3-4403-b017-785262563b0b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:27.687413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 68/132] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:27 2022] GET /identity/v3/users/61613a3a760a44eea15c4d454557a296 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:27.689098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc1dee-32f3-4403-b017-785262563b0b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.767447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba535-d621-47e9-ae2e-176ded341fdf None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:27.794880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc1dee-32f3-4403-b017-785262563b0b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:27.806239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42fcba-8474-45c1-aee2-83d736b0e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:27.807320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42fcba-8474-45c1-aee2-83d736b0e36f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:27.807753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42fcba-8474-45c1-aee2-83d736b0e36f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:27.808201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42fcba-8474-45c1-aee2-83d736b0e36f None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:27.809646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42fcba-8474-45c1-aee2-83d736b0e36f None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:27.811477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42fcba-8474-45c1-aee2-83d736b0e36f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:27.813580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 65/133] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:27 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:27.814625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42fcba-8474-45c1-aee2-83d736b0e36f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.828087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:27.837475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0baa1-4b5b-45da-81fe-5fa1a77f659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:27.838433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0baa1-4b5b-45da-81fe-5fa1a77f659f None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:27.838770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0baa1-4b5b-45da-81fe-5fa1a77f659f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:27.839049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0baa1-4b5b-45da-81fe-5fa1a77f659f None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20/users/61613a3a760a44eea15c4d454557a296/roles/e2a3f2f08d18410a954a4d6b8314562d` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:27.841575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0baa1-4b5b-45da-81fe-5fa1a77f659f None admin] RBAC: Authorizing `identity:create_grant(project_id=84b308b5ae9a4e0482f37715f327cd20, user_id=61613a3a760a44eea15c4d454557a296, role_id=e2a3f2f08d18410a954a4d6b8314562d)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:27.843128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0baa1-4b5b-45da-81fe-5fa1a77f659f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:27.862731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 69/134] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:15:27 2022] PUT /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20/users/61613a3a760a44eea15c4d454557a296/roles/e2a3f2f08d18410a954a4d6b8314562d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:27.863713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0baa1-4b5b-45da-81fe-5fa1a77f659f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:27.868200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:27.990255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0baa1-4b5b-45da-81fe-5fa1a77f659f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:27.996924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42fcba-8474-45c1-aee2-83d736b0e36f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:28.029974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fda99-eb73-4384-b801-4ae6b513ee71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:28.031075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fda99-eb73-4384-b801-4ae6b513ee71 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:28.031468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fda99-eb73-4384-b801-4ae6b513ee71 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:28.031958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fda99-eb73-4384-b801-4ae6b513ee71 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20/users/61ee072739a04036a34cf038cd44e02b/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:28.034512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fda99-eb73-4384-b801-4ae6b513ee71 None admin] RBAC: Authorizing `identity:create_grant(project_id=84b308b5ae9a4e0482f37715f327cd20, user_id=61ee072739a04036a34cf038cd44e02b, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:28.034797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a81c3-7ba6-42b9-97fc-5925aa003d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:28.035686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a81c3-7ba6-42b9-97fc-5925aa003d40 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:28.035966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a81c3-7ba6-42b9-97fc-5925aa003d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:28.036238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a81c3-7ba6-42b9-97fc-5925aa003d40 None admin] PATH_INFO: `/v3/projects/82aa67c5532c41168f1d0faa3949e3c1` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:28.036552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fda99-eb73-4384-b801-4ae6b513ee71 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:28.037552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a81c3-7ba6-42b9-97fc-5925aa003d40 None admin] RBAC: Authorizing `identity:get_project(project_id=82aa67c5532c41168f1d0faa3949e3c1)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:28.039030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a81c3-7ba6-42b9-97fc-5925aa003d40 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:28.040824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 66/135] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:27 2022] GET /identity/v3/projects/82aa67c5532c41168f1d0faa3949e3c1 => generated 312 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:28.046944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a81c3-7ba6-42b9-97fc-5925aa003d40 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:28.051519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 70/136] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:15:27 2022] PUT /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20/users/61ee072739a04036a34cf038cd44e02b/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:28.168717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a81c3-7ba6-42b9-97fc-5925aa003d40 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:28.179253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:28.207178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb066828-2f09-4a1d-a67f-498cbfe892f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:28.208110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb066828-2f09-4a1d-a67f-498cbfe892f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:28.208394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb066828-2f09-4a1d-a67f-498cbfe892f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:28.208672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb066828-2f09-4a1d-a67f-498cbfe892f5 None admin] PATH_INFO: `/v3/projects/82aa67c5532c41168f1d0faa3949e3c1/users/61613a3a760a44eea15c4d454557a296/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:28.210884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb066828-2f09-4a1d-a67f-498cbfe892f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=82aa67c5532c41168f1d0faa3949e3c1, user_id=61613a3a760a44eea15c4d454557a296, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:28.212988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb066828-2f09-4a1d-a67f-498cbfe892f5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:28.239365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 67/137] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:15:28 2022] PUT /identity/v3/projects/82aa67c5532c41168f1d0faa3949e3c1/users/61613a3a760a44eea15c4d454557a296/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:28.660148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12942256-6a87-4d0e-8473-00de3d3d79dd None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:28.660640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12942256-6a87-4d0e-8473-00de3d3d79dd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:28.660710 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12942256-6a87-4d0e-8473-00de3d3d79dd None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:28.662346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 71/138] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:28.668369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce583-0054-4725-a6f7-8f20df359429 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:28.668660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce583-0054-4725-a6f7-8f20df359429 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:28.669009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce583-0054-4725-a6f7-8f20df359429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:28.691240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ff1ce583-0054-4725-a6f7-8f20df359429 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:28.692435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ff1ce583-0054-4725-a6f7-8f20df359429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:28.781503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 68/139] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:28.818148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12942256-6a87-4d0e-8473-00de3d3d79dd None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:28.888219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:28.940997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d58d1f-00bc-4796-b2a7-ac09f9a48043 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:28.941309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d58d1f-00bc-4796-b2a7-ac09f9a48043 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:28.941911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d58d1f-00bc-4796-b2a7-ac09f9a48043 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:28.943784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 69/140] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:28.952684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb5569-899a-4338-87ec-3e78f97962e0 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:28.953002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb5569-899a-4338-87ec-3e78f97962e0 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:28.953651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb5569-899a-4338-87ec-3e78f97962e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:28.961730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12942256-6a87-4d0e-8473-00de3d3d79dd None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:28.980074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-71bb5569-899a-4338-87ec-3e78f97962e0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:28.981297 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-71bb5569-899a-4338-87ec-3e78f97962e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:29.047281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1adb6e-7562-433f-99d6-040d88182587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:29.048094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1adb6e-7562-433f-99d6-040d88182587 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:29.048427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1adb6e-7562-433f-99d6-040d88182587 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:29.048709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1adb6e-7562-433f-99d6-040d88182587 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:29.050095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1adb6e-7562-433f-99d6-040d88182587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:29.051562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1adb6e-7562-433f-99d6-040d88182587 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:29.060075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 72/141] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:29.066000 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1adb6e-7562-433f-99d6-040d88182587 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:29.105099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 70/142] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:29.143571 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb5569-899a-4338-87ec-3e78f97962e0 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:29.201868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:29.309081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1adb6e-7562-433f-99d6-040d88182587 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:29.342247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:29.372835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb5569-899a-4338-87ec-3e78f97962e0 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:29.379120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2a68f-5dbd-4d42-afe4-3a5fed59ee72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:29.385943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2a68f-5dbd-4d42-afe4-3a5fed59ee72 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:29.386510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2a68f-5dbd-4d42-afe4-3a5fed59ee72 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:29.387024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2a68f-5dbd-4d42-afe4-3a5fed59ee72 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:29.387973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2a68f-5dbd-4d42-afe4-3a5fed59ee72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:29.389735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2a68f-5dbd-4d42-afe4-3a5fed59ee72 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:29.423678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 73/143] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:15:29 2022] POST /identity/v3/users => generated 283 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:29.447964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5cd1b-b331-478d-a131-30184d048504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:29.449154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5cd1b-b331-478d-a131-30184d048504 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:29.449668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5cd1b-b331-478d-a131-30184d048504 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:29.450149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5cd1b-b331-478d-a131-30184d048504 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:29.474324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5cd1b-b331-478d-a131-30184d048504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-61e5cd1b-b331-478d-a131-30184d048504 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:29.486188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:29.487153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:29.487882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:29.495709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 71/144] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:29.501850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2a68f-5dbd-4d42-afe4-3a5fed59ee72 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:29.651756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2a68f-5dbd-4d42-afe4-3a5fed59ee72 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:29.691595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375620a3-7aa0-4b77-8473-5c1a8686672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:29.692214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375620a3-7aa0-4b77-8473-5c1a8686672b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:29.692495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375620a3-7aa0-4b77-8473-5c1a8686672b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:29.692778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375620a3-7aa0-4b77-8473-5c1a8686672b None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:29.693591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375620a3-7aa0-4b77-8473-5c1a8686672b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:29.693902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375620a3-7aa0-4b77-8473-5c1a8686672b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:29.695366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375620a3-7aa0-4b77-8473-5c1a8686672b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:29.705562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 74/145] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:29.714062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5cd1b-b331-478d-a131-30184d048504 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:29.847578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:29.865708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5cd1b-b331-478d-a131-30184d048504 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:29.913445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f94252-e219-4c18-b64d-5c66ef18f6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:29.914324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f94252-e219-4c18-b64d-5c66ef18f6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:29.914603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f94252-e219-4c18-b64d-5c66ef18f6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:29.914877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f94252-e219-4c18-b64d-5c66ef18f6d1 None admin] PATH_INFO: `/v3/users/7d91e07c08224d54af44714a692b4973` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:29.916702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f94252-e219-4c18-b64d-5c66ef18f6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=7d91e07c08224d54af44714a692b4973)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:29.918218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f94252-e219-4c18-b64d-5c66ef18f6d1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:29.921154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 72/146] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:29 2022] GET /identity/v3/users/7d91e07c08224d54af44714a692b4973 => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:29.925870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375620a3-7aa0-4b77-8473-5c1a8686672b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:29.967190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:30.052723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375620a3-7aa0-4b77-8473-5c1a8686672b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:30.101604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a5c7b-315e-4a7d-a335-7c20f9009438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:30.102495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a5c7b-315e-4a7d-a335-7c20f9009438 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:30.102983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a5c7b-315e-4a7d-a335-7c20f9009438 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:30.103174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a5c7b-315e-4a7d-a335-7c20f9009438 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:30.104472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a5c7b-315e-4a7d-a335-7c20f9009438 None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:30.105954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a5c7b-315e-4a7d-a335-7c20f9009438 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:30.119317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 75/147] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:29 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:30.119486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f94252-e219-4c18-b64d-5c66ef18f6d1 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:30.256932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f94252-e219-4c18-b64d-5c66ef18f6d1 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:30.300565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de69df-c73f-45fa-9e4a-81ba794d38e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:30.301837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de69df-c73f-45fa-9e4a-81ba794d38e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:30.302310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de69df-c73f-45fa-9e4a-81ba794d38e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:30.302769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de69df-c73f-45fa-9e4a-81ba794d38e6 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20/users/7d91e07c08224d54af44714a692b4973/roles/2acd8a15a63a4261b1ae12f09d28722f` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:30.306825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de69df-c73f-45fa-9e4a-81ba794d38e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=84b308b5ae9a4e0482f37715f327cd20, user_id=7d91e07c08224d54af44714a692b4973, role_id=2acd8a15a63a4261b1ae12f09d28722f)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:30.308298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de69df-c73f-45fa-9e4a-81ba794d38e6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:30.333554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 73/148] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:15:30 2022] PUT /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20/users/7d91e07c08224d54af44714a692b4973/roles/2acd8a15a63a4261b1ae12f09d28722f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:30.751434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5473ee1-b02d-43fd-8250-222675cfda43 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:30.751922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5473ee1-b02d-43fd-8250-222675cfda43 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:30.751993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5473ee1-b02d-43fd-8250-222675cfda43 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:30.753409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 76/149] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:30.759098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0d251-e17f-4ac2-b397-489bf762cea5 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:30.759661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0d251-e17f-4ac2-b397-489bf762cea5 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:30.760151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0d251-e17f-4ac2-b397-489bf762cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:30.781373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-1aa0d251-e17f-4ac2-b397-489bf762cea5 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:30.782878 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1aa0d251-e17f-4ac2-b397-489bf762cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:30.797384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:30.879205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 74/150] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:30.922756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5473ee1-b02d-43fd-8250-222675cfda43 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:31.042062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:31.047003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5473ee1-b02d-43fd-8250-222675cfda43 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:31.083388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ed3a9-3a92-4e5d-91fb-f646a384d606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:31.084563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ed3a9-3a92-4e5d-91fb-f646a384d606 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:31.085076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ed3a9-3a92-4e5d-91fb-f646a384d606 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:31.085548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ed3a9-3a92-4e5d-91fb-f646a384d606 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:31.097889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ed3a9-3a92-4e5d-91fb-f646a384d606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-487ed3a9-3a92-4e5d-91fb-f646a384d606 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:31.106344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:31.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:31.107930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:31.110315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 77/151] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:31.116690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0d251-e17f-4ac2-b397-489bf762cea5 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:31.233305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0d251-e17f-4ac2-b397-489bf762cea5 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:31.280421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252f458-2f7e-4785-95f2-a4e1cf834657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:31.281924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252f458-2f7e-4785-95f2-a4e1cf834657 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:31.282414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252f458-2f7e-4785-95f2-a4e1cf834657 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:31.282881 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252f458-2f7e-4785-95f2-a4e1cf834657 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:31.283835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252f458-2f7e-4785-95f2-a4e1cf834657 None admin] RBAC: Adding query filter params (name=member) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:31.284333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252f458-2f7e-4785-95f2-a4e1cf834657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:31.285966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252f458-2f7e-4785-95f2-a4e1cf834657 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:31.295714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 75/152] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:31.301256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ed3a9-3a92-4e5d-91fb-f646a384d606 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:31.350273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e95121-63c5-46ad-be49-eab8f7558782 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:31.350972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e95121-63c5-46ad-be49-eab8f7558782 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:31.351458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e95121-63c5-46ad-be49-eab8f7558782 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:31.353387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 76/153] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:15:31.358830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b156783-16cd-4f95-9a56-d06313a86b13 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:31.359364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b156783-16cd-4f95-9a56-d06313a86b13 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:31.359830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b156783-16cd-4f95-9a56-d06313a86b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:31.392523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-9b156783-16cd-4f95-9a56-d06313a86b13 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:31.401453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-9b156783-16cd-4f95-9a56-d06313a86b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:31.423657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:31.531164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ed3a9-3a92-4e5d-91fb-f646a384d606 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:31.555088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 77/154] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:31.598790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b156783-16cd-4f95-9a56-d06313a86b13 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:31.614204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93e494-d87a-485c-a5c7-85dc617c310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:31.615108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93e494-d87a-485c-a5c7-85dc617c310c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:31.615386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93e494-d87a-485c-a5c7-85dc617c310c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:31.615654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93e494-d87a-485c-a5c7-85dc617c310c None admin] PATH_INFO: `/v3/users/61613a3a760a44eea15c4d454557a296` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:31.616856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93e494-d87a-485c-a5c7-85dc617c310c None admin] RBAC: Authorizing `identity:get_user(user_id=61613a3a760a44eea15c4d454557a296)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:31.618420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93e494-d87a-485c-a5c7-85dc617c310c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:31.620180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 78/155] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:31 2022] GET /identity/v3/users/61613a3a760a44eea15c4d454557a296 => generated 275 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:31.625999 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93e494-d87a-485c-a5c7-85dc617c310c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:31.693033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:31.817514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93e494-d87a-485c-a5c7-85dc617c310c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:31.823444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b156783-16cd-4f95-9a56-d06313a86b13 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:31.878436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e9925-07f0-4163-99f1-cf3c5ce333de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:31.879601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e9925-07f0-4163-99f1-cf3c5ce333de None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:31.880102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e9925-07f0-4163-99f1-cf3c5ce333de None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:31.880571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e9925-07f0-4163-99f1-cf3c5ce333de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:31.906493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3ccfd-1a95-4c53-870c-41df3bbc7b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:31.907314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3ccfd-1a95-4c53-870c-41df3bbc7b55 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:31.907597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3ccfd-1a95-4c53-870c-41df3bbc7b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:31.907873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3ccfd-1a95-4c53-870c-41df3bbc7b55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:31.930165 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:31.942587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3ccfd-1a95-4c53-870c-41df3bbc7b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-c8f3ccfd-1a95-4c53-870c-41df3bbc7b55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:31.966563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:31.967665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:31.968491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:31.969932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 78/156] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:15:31.971273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740e1f2-ca68-4524-9ddb-800e69df50e9 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:31.971558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740e1f2-ca68-4524-9ddb-800e69df50e9 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:31.971834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740e1f2-ca68-4524-9ddb-800e69df50e9 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:31.974608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 79/157] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:31.981503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f8113-7b25-472d-a7b2-de5f773267cf None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:31.981788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f8113-7b25-472d-a7b2-de5f773267cf None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:31.982064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f8113-7b25-472d-a7b2-de5f773267cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:32.016194 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-080f8113-7b25-472d-a7b2-de5f773267cf None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:32.018646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-080f8113-7b25-472d-a7b2-de5f773267cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:32.032790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e9925-07f0-4163-99f1-cf3c5ce333de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:32.034320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e9925-07f0-4163-99f1-cf3c5ce333de None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:32.089895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:32.171573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 80/158] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:32.172465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f8113-7b25-472d-a7b2-de5f773267cf None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:32.289540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 79/159] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:15:32.291167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d083b-dfdb-408e-83c0-1521c2d6d554 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:32.291467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d083b-dfdb-408e-83c0-1521c2d6d554 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:32.291749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d083b-dfdb-408e-83c0-1521c2d6d554 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:32.293080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 80/160] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:32.293792 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d083b-dfdb-408e-83c0-1521c2d6d554 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:32.357315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f8113-7b25-472d-a7b2-de5f773267cf None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:32.399529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:32.413258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5bbd7-40fe-430a-abce-ee0e3a82f513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:32.413965 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5bbd7-40fe-430a-abce-ee0e3a82f513 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:32.414251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5bbd7-40fe-430a-abce-ee0e3a82f513 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:32.414530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5bbd7-40fe-430a-abce-ee0e3a82f513 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:32.415859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5bbd7-40fe-430a-abce-ee0e3a82f513 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:32.416169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5bbd7-40fe-430a-abce-ee0e3a82f513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:32.417627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5bbd7-40fe-430a-abce-ee0e3a82f513 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:32.437179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 81/161] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:32.438466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e47d1-761c-45b3-9ac5-ffe7ce1d1273 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:32.438751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e47d1-761c-45b3-9ac5-ffe7ce1d1273 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:32.439022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e47d1-761c-45b3-9ac5-ffe7ce1d1273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:32.463474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-832e47d1-761c-45b3-9ac5-ffe7ce1d1273 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:32.464662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-832e47d1-761c-45b3-9ac5-ffe7ce1d1273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:32.553476 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 82/162] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:32.554339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e47d1-761c-45b3-9ac5-ffe7ce1d1273 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:32.642201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d083b-dfdb-408e-83c0-1521c2d6d554 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:32.668969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e47d1-761c-45b3-9ac5-ffe7ce1d1273 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:32.691658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:32.695450 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:32.707002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908007c-8f79-445b-b45c-9cc1c7dcb332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:32.708129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908007c-8f79-445b-b45c-9cc1c7dcb332 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:32.708549 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908007c-8f79-445b-b45c-9cc1c7dcb332 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:32.709002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908007c-8f79-445b-b45c-9cc1c7dcb332 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:32.710429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908007c-8f79-445b-b45c-9cc1c7dcb332 None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:32.712725 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908007c-8f79-445b-b45c-9cc1c7dcb332 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:32.714938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 83/163] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:32 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:32.715693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db164c-757c-441d-bbf3-de6292a5aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:32.716042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908007c-8f79-445b-b45c-9cc1c7dcb332 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:32.716549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db164c-757c-441d-bbf3-de6292a5aeff None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:32.716907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db164c-757c-441d-bbf3-de6292a5aeff None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:32.717144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db164c-757c-441d-bbf3-de6292a5aeff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:32.738864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db164c-757c-441d-bbf3-de6292a5aeff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-d7db164c-757c-441d-bbf3-de6292a5aeff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:32.753809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:32.754660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:32.755425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:32.761022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 81/164] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:15:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:32.761868 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db164c-757c-441d-bbf3-de6292a5aeff None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:32.893462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908007c-8f79-445b-b45c-9cc1c7dcb332 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:32.947383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03c0db-269b-4112-a596-43ebe4c55d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:32.948266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03c0db-269b-4112-a596-43ebe4c55d2f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:32.948547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03c0db-269b-4112-a596-43ebe4c55d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:32.948819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03c0db-269b-4112-a596-43ebe4c55d2f None admin] PATH_INFO: `/v3/users/61ee072739a04036a34cf038cd44e02b` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:32.952913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03c0db-269b-4112-a596-43ebe4c55d2f None admin] RBAC: Authorizing `identity:get_user(user_id=61ee072739a04036a34cf038cd44e02b)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:32.954403 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03c0db-269b-4112-a596-43ebe4c55d2f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:32.956097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 84/165] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:32 2022] GET /identity/v3/users/61ee072739a04036a34cf038cd44e02b => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:32.956880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03c0db-269b-4112-a596-43ebe4c55d2f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:32.992378 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db164c-757c-441d-bbf3-de6292a5aeff None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:33.058701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae0640-0b47-479c-b907-444ddade0d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:33.059946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae0640-0b47-479c-b907-444ddade0d0f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:33.060678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae0640-0b47-479c-b907-444ddade0d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:33.061933 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae0640-0b47-479c-b907-444ddade0d0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:33.077223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:33.080960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae0640-0b47-479c-b907-444ddade0d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-33ae0640-0b47-479c-b907-444ddade0d0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:33.091722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:33.092584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:33.093830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 82/166] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:33.097754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae0640-0b47-479c-b907-444ddade0d0f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:33.161632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03c0db-269b-4112-a596-43ebe4c55d2f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:33.228787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7817d0c-5a61-4c73-a99d-aaf7bd4f7e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:33.229640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7817d0c-5a61-4c73-a99d-aaf7bd4f7e53 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:33.230061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7817d0c-5a61-4c73-a99d-aaf7bd4f7e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:33.230348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7817d0c-5a61-4c73-a99d-aaf7bd4f7e53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:33.231194 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7817d0c-5a61-4c73-a99d-aaf7bd4f7e53 None admin] RBAC: Adding query filter params (role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=84b308b5ae9a4e0482f37715f327cd20, user.id=61613a3a760a44eea15c4d454557a296) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:33.231509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7817d0c-5a61-4c73-a99d-aaf7bd4f7e53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:33.233016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7817d0c-5a61-4c73-a99d-aaf7bd4f7e53 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:33.234527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:33.241069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 85/167] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:32 2022] GET /identity/v3/role_assignments?user.id=61613a3a760a44eea15c4d454557a296&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=8de5789832e84f9388b1295d479fca6e => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:33.242416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7817d0c-5a61-4c73-a99d-aaf7bd4f7e53 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:33.443054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae0640-0b47-479c-b907-444ddade0d0f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:33.445429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7817d0c-5a61-4c73-a99d-aaf7bd4f7e53 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:33.485660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:33.528153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fdcbf-9199-46fe-99a1-4007dede754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:33.529328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fdcbf-9199-46fe-99a1-4007dede754c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:33.529813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fdcbf-9199-46fe-99a1-4007dede754c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:33.530327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fdcbf-9199-46fe-99a1-4007dede754c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:33.530496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd38768-7e15-4364-92ba-cfc978d66f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:33.531155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd38768-7e15-4364-92ba-cfc978d66f8b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:33.531465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd38768-7e15-4364-92ba-cfc978d66f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:33.531716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd38768-7e15-4364-92ba-cfc978d66f8b None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:33.536891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd38768-7e15-4364-92ba-cfc978d66f8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:33.536891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd38768-7e15-4364-92ba-cfc978d66f8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:33.536891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd38768-7e15-4364-92ba-cfc978d66f8b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:33.552017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 83/168] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:15:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:33.552935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd38768-7e15-4364-92ba-cfc978d66f8b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:33.627807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fdcbf-9199-46fe-99a1-4007dede754c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:33.629364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fdcbf-9199-46fe-99a1-4007dede754c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:33.762825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 86/169] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:33.763818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fdcbf-9199-46fe-99a1-4007dede754c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:33.801315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:33.912264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fdcbf-9199-46fe-99a1-4007dede754c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:33.934648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd38768-7e15-4364-92ba-cfc978d66f8b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:33.963743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:33.967652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652631f9-21dd-4e82-8a9b-93dd355c3240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:33.968556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652631f9-21dd-4e82-8a9b-93dd355c3240 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:33.968840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652631f9-21dd-4e82-8a9b-93dd355c3240 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:33.969173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652631f9-21dd-4e82-8a9b-93dd355c3240 None admin] PATH_INFO: `/v3/users/61613a3a760a44eea15c4d454557a296` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:33.970686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652631f9-21dd-4e82-8a9b-93dd355c3240 None admin] RBAC: Authorizing `identity:get_user(user_id=61613a3a760a44eea15c4d454557a296)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:33.972179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652631f9-21dd-4e82-8a9b-93dd355c3240 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:33.974158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 87/170] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:33 2022] GET /identity/v3/users/61613a3a760a44eea15c4d454557a296 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:33.974942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652631f9-21dd-4e82-8a9b-93dd355c3240 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:34.027747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da8f5e-136b-463a-aa7e-7227e897c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:34.028435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8f5e-136b-463a-aa7e-7227e897c291 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.028714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8f5e-136b-463a-aa7e-7227e897c291 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.029028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8f5e-136b-463a-aa7e-7227e897c291 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.029794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da8f5e-136b-463a-aa7e-7227e897c291 None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:34.030105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da8f5e-136b-463a-aa7e-7227e897c291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:34.031487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da8f5e-136b-463a-aa7e-7227e897c291 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:34.040604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 84/171] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:34.041489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da8f5e-136b-463a-aa7e-7227e897c291 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:34.077219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:34.085485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652631f9-21dd-4e82-8a9b-93dd355c3240 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:34.146168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da8f5e-136b-463a-aa7e-7227e897c291 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:34.161835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07e89f-3682-42fd-b1be-ecb40217eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:34.162856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e89f-3682-42fd-b1be-ecb40217eb64 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.163144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e89f-3682-42fd-b1be-ecb40217eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.163421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e89f-3682-42fd-b1be-ecb40217eb64 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.166643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07e89f-3682-42fd-b1be-ecb40217eb64 None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:34.168159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07e89f-3682-42fd-b1be-ecb40217eb64 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:34.170186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 88/172] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:33 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:34.171501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77f74b-b815-4ca8-b242-d371bf446af1 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.171783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77f74b-b815-4ca8-b242-d371bf446af1 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.172061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77f74b-b815-4ca8-b242-d371bf446af1 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.182528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 89/173] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:34.183320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77f74b-b815-4ca8-b242-d371bf446af1 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:34.195713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf99a7b-f16e-48b8-a3f1-4b8023d2b461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:34.197031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf99a7b-f16e-48b8-a3f1-4b8023d2b461 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.197318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf99a7b-f16e-48b8-a3f1-4b8023d2b461 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.197593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf99a7b-f16e-48b8-a3f1-4b8023d2b461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.289964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:34.340728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf99a7b-f16e-48b8-a3f1-4b8023d2b461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:34.342257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf99a7b-f16e-48b8-a3f1-4b8023d2b461 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:34.388415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77f74b-b815-4ca8-b242-d371bf446af1 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:34.434421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb13ec-1264-412d-bca2-aac16d945b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:34.435299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb13ec-1264-412d-bca2-aac16d945b52 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.435575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb13ec-1264-412d-bca2-aac16d945b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.435844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb13ec-1264-412d-bca2-aac16d945b52 None admin] PATH_INFO: `/v3/users/61613a3a760a44eea15c4d454557a296` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.437681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb13ec-1264-412d-bca2-aac16d945b52 None admin] RBAC: Authorizing `identity:get_user(user_id=61613a3a760a44eea15c4d454557a296)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:34.439146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb13ec-1264-412d-bca2-aac16d945b52 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:34.440810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 90/174] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:34 2022] GET /identity/v3/users/61613a3a760a44eea15c4d454557a296 => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:34.445839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb13ec-1264-412d-bca2-aac16d945b52 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:34.453206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 85/175] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:34.453206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ecbe-2255-4286-bd4d-4e73aac2065a None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.453206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ecbe-2255-4286-bd4d-4e73aac2065a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.453206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ecbe-2255-4286-bd4d-4e73aac2065a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.470927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-44b6ecbe-2255-4286-bd4d-4e73aac2065a None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:34.472245 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-44b6ecbe-2255-4286-bd4d-4e73aac2065a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:34.476202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:34.554469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 86/176] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:34.556304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b182da-a571-49ec-b08b-e481193ddae0 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.556599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b182da-a571-49ec-b08b-e481193ddae0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.556949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b182da-a571-49ec-b08b-e481193ddae0 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.559169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 87/177] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:34.560666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1eeba-fda2-4083-b4a4-6bcf7d5ed3ca None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.561134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1eeba-fda2-4083-b4a4-6bcf7d5ed3ca None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.561592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1eeba-fda2-4083-b4a4-6bcf7d5ed3ca None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.567396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 88/178] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:34.567396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1eeba-fda2-4083-b4a4-6bcf7d5ed3ca None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:34.568590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:34.579461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb13ec-1264-412d-bca2-aac16d945b52 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:34.618748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af6b3-40c1-4600-8836-55c8625eb78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:34.619521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af6b3-40c1-4600-8836-55c8625eb78d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.619802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af6b3-40c1-4600-8836-55c8625eb78d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.620074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af6b3-40c1-4600-8836-55c8625eb78d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.621197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155af6b3-40c1-4600-8836-55c8625eb78d None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=84b308b5ae9a4e0482f37715f327cd20, user.id=61ee072739a04036a34cf038cd44e02b) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:34.621506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155af6b3-40c1-4600-8836-55c8625eb78d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:34.622963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155af6b3-40c1-4600-8836-55c8625eb78d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:34.628983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 91/179] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:34 2022] GET /identity/v3/role_assignments?user.id=61ee072739a04036a34cf038cd44e02b&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:34.629772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af6b3-40c1-4600-8836-55c8625eb78d None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:34.698716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1eeba-fda2-4083-b4a4-6bcf7d5ed3ca None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:34.741278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848159d-c737-4714-bb46-48b23593e5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:34.742218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848159d-c737-4714-bb46-48b23593e5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.742498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848159d-c737-4714-bb46-48b23593e5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.744016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848159d-c737-4714-bb46-48b23593e5f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.751587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:34.765765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af6b3-40c1-4600-8836-55c8625eb78d None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:34.804584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2114f-65a1-4c81-b59e-908df6b93413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:34.805479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2114f-65a1-4c81-b59e-908df6b93413 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.806398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2114f-65a1-4c81-b59e-908df6b93413 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.806947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2114f-65a1-4c81-b59e-908df6b93413 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.808068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2114f-65a1-4c81-b59e-908df6b93413 None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:34.809576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2114f-65a1-4c81-b59e-908df6b93413 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:34.811516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 92/180] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:34 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:34.812596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cc787-5172-46f8-9c3a-5f48714131d7 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.812900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cc787-5172-46f8-9c3a-5f48714131d7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.813180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cc787-5172-46f8-9c3a-5f48714131d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.827546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848159d-c737-4714-bb46-48b23593e5f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:34.829058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848159d-c737-4714-bb46-48b23593e5f0 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:34.830498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-348cc787-5172-46f8-9c3a-5f48714131d7 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:34.831169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:34.831674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-348cc787-5172-46f8-9c3a-5f48714131d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:34.923266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 93/181] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:34.924569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4b9fe-ec8f-4141-9f27-1e0eedcea447 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:34.924862 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4b9fe-ec8f-4141-9f27-1e0eedcea447 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:34.925207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4b9fe-ec8f-4141-9f27-1e0eedcea447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:34.941144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-32c4b9fe-ec8f-4141-9f27-1e0eedcea447 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:34.942319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-32c4b9fe-ec8f-4141-9f27-1e0eedcea447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:34.965929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 89/182] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:34.965929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848159d-c737-4714-bb46-48b23593e5f0 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.011159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 94/183] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:35.012702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83901253-4407-4c5d-80bb-5c201ee7e168 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.013008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83901253-4407-4c5d-80bb-5c201ee7e168 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.013287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83901253-4407-4c5d-80bb-5c201ee7e168 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.017380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 95/184] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:35.017380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83901253-4407-4c5d-80bb-5c201ee7e168 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.040985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:35.063695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848159d-c737-4714-bb46-48b23593e5f0 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.065418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:35.098935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae2d94-c4fb-451b-a018-f8a1d026a09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.099725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae2d94-c4fb-451b-a018-f8a1d026a09a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.100121 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae2d94-c4fb-451b-a018-f8a1d026a09a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.100680 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae2d94-c4fb-451b-a018-f8a1d026a09a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.111659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae2d94-c4fb-451b-a018-f8a1d026a09a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-62ae2d94-c4fb-451b-a018-f8a1d026a09a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:35.119329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:35.120195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:35.121450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 90/185] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:35.123416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae2d94-c4fb-451b-a018-f8a1d026a09a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.125197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83901253-4407-4c5d-80bb-5c201ee7e168 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.159818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a466e-9813-45ec-b12c-6b5f1c0fcd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.160469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a466e-9813-45ec-b12c-6b5f1c0fcd5e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.160752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a466e-9813-45ec-b12c-6b5f1c0fcd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.161058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a466e-9813-45ec-b12c-6b5f1c0fcd5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.161891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a466e-9813-45ec-b12c-6b5f1c0fcd5e None admin] RBAC: Adding query filter params (role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=84b308b5ae9a4e0482f37715f327cd20, user.id=61613a3a760a44eea15c4d454557a296) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:35.162200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a466e-9813-45ec-b12c-6b5f1c0fcd5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:35.163653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a466e-9813-45ec-b12c-6b5f1c0fcd5e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:35.176985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 96/186] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/role_assignments?user.id=61613a3a760a44eea15c4d454557a296&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:35.176985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a466e-9813-45ec-b12c-6b5f1c0fcd5e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.250534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae2d94-c4fb-451b-a018-f8a1d026a09a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.288346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:35.289014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320f0f1-19d7-46d9-9f07-36524d134e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.289974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f0f1-19d7-46d9-9f07-36524d134e28 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.290345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f0f1-19d7-46d9-9f07-36524d134e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.290686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f0f1-19d7-46d9-9f07-36524d134e28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.302479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320f0f1-19d7-46d9-9f07-36524d134e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:35.310414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a466e-9813-45ec-b12c-6b5f1c0fcd5e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-6320f0f1-19d7-46d9-9f07-36524d134e28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:35.310974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:35.311844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:35.312711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 91/187] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:35.314194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70eff3-24c3-4173-9e1d-c2b498c8bf51 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.314194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70eff3-24c3-4173-9e1d-c2b498c8bf51 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.314194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70eff3-24c3-4173-9e1d-c2b498c8bf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.333405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-9d70eff3-24c3-4173-9e1d-c2b498c8bf51 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:35.334628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9d70eff3-24c3-4173-9e1d-c2b498c8bf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:35.353671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:35.359286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66b08b-c999-46f3-8d72-f25b900b4c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.360164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66b08b-c999-46f3-8d72-f25b900b4c35 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.360444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66b08b-c999-46f3-8d72-f25b900b4c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.360718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66b08b-c999-46f3-8d72-f25b900b4c35 None admin] PATH_INFO: `/v3/projects/82aa67c5532c41168f1d0faa3949e3c1` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.361967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66b08b-c999-46f3-8d72-f25b900b4c35 None admin] RBAC: Authorizing `identity:get_project(project_id=82aa67c5532c41168f1d0faa3949e3c1)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:35.363458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66b08b-c999-46f3-8d72-f25b900b4c35 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:35.365858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 97/188] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/projects/82aa67c5532c41168f1d0faa3949e3c1 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:35.370799 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66b08b-c999-46f3-8d72-f25b900b4c35 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.420342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 92/189] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:35.421357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70eff3-24c3-4173-9e1d-c2b498c8bf51 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.494311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66b08b-c999-46f3-8d72-f25b900b4c35 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.527322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70eff3-24c3-4173-9e1d-c2b498c8bf51 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.529125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892ca1-c402-417f-a991-7f23cd0ca91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.529922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892ca1-c402-417f-a991-7f23cd0ca91e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.530203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892ca1-c402-417f-a991-7f23cd0ca91e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.530474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892ca1-c402-417f-a991-7f23cd0ca91e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.531642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892ca1-c402-417f-a991-7f23cd0ca91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:35.533146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892ca1-c402-417f-a991-7f23cd0ca91e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:35.534774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 98/190] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:35.537054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892ca1-c402-417f-a991-7f23cd0ca91e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.556428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:35.560830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30b192-432d-4733-a3bb-1549b7b07d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.561527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30b192-432d-4733-a3bb-1549b7b07d3d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.561800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30b192-432d-4733-a3bb-1549b7b07d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.562072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30b192-432d-4733-a3bb-1549b7b07d3d None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.562817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30b192-432d-4733-a3bb-1549b7b07d3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:35.563119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30b192-432d-4733-a3bb-1549b7b07d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:35.564536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30b192-432d-4733-a3bb-1549b7b07d3d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:35.573293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 93/191] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:35.573902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30b192-432d-4733-a3bb-1549b7b07d3d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.589823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:35.651069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892ca1-c402-417f-a991-7f23cd0ca91e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.674615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30b192-432d-4733-a3bb-1549b7b07d3d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.683822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e65c5-b7c3-46f4-bed9-60da74e4bbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.684487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e65c5-b7c3-46f4-bed9-60da74e4bbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.684769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e65c5-b7c3-46f4-bed9-60da74e4bbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.685065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e65c5-b7c3-46f4-bed9-60da74e4bbb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.685808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e65c5-b7c3-46f4-bed9-60da74e4bbb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:35.686107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e65c5-b7c3-46f4-bed9-60da74e4bbb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:35.687477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e65c5-b7c3-46f4-bed9-60da74e4bbb4 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:35.695377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 99/192] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:35.695995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e65c5-b7c3-46f4-bed9-60da74e4bbb4 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.711122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc46e8b-63ec-40de-979a-2f5a76531f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.712518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc46e8b-63ec-40de-979a-2f5a76531f2a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.712805 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc46e8b-63ec-40de-979a-2f5a76531f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.713118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc46e8b-63ec-40de-979a-2f5a76531f2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.713953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc46e8b-63ec-40de-979a-2f5a76531f2a None admin] RBAC: Adding query filter params (role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=82aa67c5532c41168f1d0faa3949e3c1, user.id=61613a3a760a44eea15c4d454557a296) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:35.714266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc46e8b-63ec-40de-979a-2f5a76531f2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:35.715720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc46e8b-63ec-40de-979a-2f5a76531f2a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:35.721498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 94/193] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/role_assignments?user.id=61613a3a760a44eea15c4d454557a296&scope.project.id=82aa67c5532c41168f1d0faa3949e3c1&role.id=8de5789832e84f9388b1295d479fca6e => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:35.722264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc46e8b-63ec-40de-979a-2f5a76531f2a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.777945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:35.802436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e65c5-b7c3-46f4-bed9-60da74e4bbb4 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.815423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:35.840856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc46e8b-63ec-40de-979a-2f5a76531f2a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:35.841780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7fb2-9ba8-4720-9172-fecbfc30b67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.842588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7fb2-9ba8-4720-9172-fecbfc30b67e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.842869 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7fb2-9ba8-4720-9172-fecbfc30b67e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.843140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7fb2-9ba8-4720-9172-fecbfc30b67e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.856983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c7fb2-9ba8-4720-9172-fecbfc30b67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-328c7fb2-9ba8-4720-9172-fecbfc30b67e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:35.864959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:35.865886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:35.866659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 100/194] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:35.868526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7fb2-9ba8-4720-9172-fecbfc30b67e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:35.887499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d92b9e-8c5c-40e2-89ee-28c62b8cea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:35.888907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d92b9e-8c5c-40e2-89ee-28c62b8cea71 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:35.889393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d92b9e-8c5c-40e2-89ee-28c62b8cea71 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:35.889996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d92b9e-8c5c-40e2-89ee-28c62b8cea71 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:35.891242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d92b9e-8c5c-40e2-89ee-28c62b8cea71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:35.892994 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d92b9e-8c5c-40e2-89ee-28c62b8cea71 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:35.918916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 95/195] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:15:35 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:35.919786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d92b9e-8c5c-40e2-89ee-28c62b8cea71 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:36.003607 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7fb2-9ba8-4720-9172-fecbfc30b67e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:36.051073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054927d7-5c27-42f1-affb-d76f3159eafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:36.051989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054927d7-5c27-42f1-affb-d76f3159eafe None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:36.052265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054927d7-5c27-42f1-affb-d76f3159eafe None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:36.052553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054927d7-5c27-42f1-affb-d76f3159eafe None admin] PATH_INFO: `/v3/users/16222bf7be05422e98ecbd86571482e9` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:36.056798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d92b9e-8c5c-40e2-89ee-28c62b8cea71 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:36.061695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:36.069257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054927d7-5c27-42f1-affb-d76f3159eafe None admin] RBAC: Authorizing `identity:get_user(user_id=16222bf7be05422e98ecbd86571482e9)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:36.070796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054927d7-5c27-42f1-affb-d76f3159eafe None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:36.073326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 101/196] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/users/16222bf7be05422e98ecbd86571482e9 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:36.074121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054927d7-5c27-42f1-affb-d76f3159eafe None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:36.098250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2693-c0c2-46ad-af3f-bb3264fa38a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:36.099145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2693-c0c2-46ad-af3f-bb3264fa38a6 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:36.099427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2693-c0c2-46ad-af3f-bb3264fa38a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:36.099699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2693-c0c2-46ad-af3f-bb3264fa38a6 None admin] PATH_INFO: `/v3/users/16222bf7be05422e98ecbd86571482e9` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:36.100963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a2693-c0c2-46ad-af3f-bb3264fa38a6 None admin] RBAC: Authorizing `identity:get_user(user_id=16222bf7be05422e98ecbd86571482e9)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:36.102964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a2693-c0c2-46ad-af3f-bb3264fa38a6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:36.105286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 96/197] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:35 2022] GET /identity/v3/users/16222bf7be05422e98ecbd86571482e9 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:36.105881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2693-c0c2-46ad-af3f-bb3264fa38a6 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:36.108774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:36.193392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054927d7-5c27-42f1-affb-d76f3159eafe None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:36.228339 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2693-c0c2-46ad-af3f-bb3264fa38a6 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:36.236500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b93113-f15d-4b1d-b655-4e555c8f8e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:36.237527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b93113-f15d-4b1d-b655-4e555c8f8e85 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:36.237807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b93113-f15d-4b1d-b655-4e555c8f8e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:36.238084 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b93113-f15d-4b1d-b655-4e555c8f8e85 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:36.238832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b93113-f15d-4b1d-b655-4e555c8f8e85 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:36.239138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b93113-f15d-4b1d-b655-4e555c8f8e85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:36.240641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b93113-f15d-4b1d-b655-4e555c8f8e85 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:36.250739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 102/198] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:36.251397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b93113-f15d-4b1d-b655-4e555c8f8e85 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:36.268492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1416-e8f2-4ec5-8888-199ac6a90490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:36.269208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1416-e8f2-4ec5-8888-199ac6a90490 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:36.269491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1416-e8f2-4ec5-8888-199ac6a90490 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:36.269764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1416-e8f2-4ec5-8888-199ac6a90490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:36.324839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:36.347775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb1416-e8f2-4ec5-8888-199ac6a90490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:36.349651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb1416-e8f2-4ec5-8888-199ac6a90490 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:36.361225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:36.383281 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b93113-f15d-4b1d-b655-4e555c8f8e85 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:36.429560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da998ec0-0fa9-4d09-9846-149dad78c03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:36.431123 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da998ec0-0fa9-4d09-9846-149dad78c03c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:36.431598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da998ec0-0fa9-4d09-9846-149dad78c03c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:36.432816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da998ec0-0fa9-4d09-9846-149dad78c03c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:36.464031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 97/199] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:36.464937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1416-e8f2-4ec5-8888-199ac6a90490 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:36.530564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da998ec0-0fa9-4d09-9846-149dad78c03c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:36.534369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da998ec0-0fa9-4d09-9846-149dad78c03c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:36.570304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1416-e8f2-4ec5-8888-199ac6a90490 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:36.572120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:36.613096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dc759-b145-4922-b9ec-8b062c409ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:36.614148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dc759-b145-4922-b9ec-8b062c409ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:36.614506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dc759-b145-4922-b9ec-8b062c409ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:36.614850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dc759-b145-4922-b9ec-8b062c409ed0 None admin] PATH_INFO: `/v3/users/7d91e07c08224d54af44714a692b4973` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:36.615606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:36.617052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dc759-b145-4922-b9ec-8b062c409ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=7d91e07c08224d54af44714a692b4973)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:36.619647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dc759-b145-4922-b9ec-8b062c409ed0 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:36.622756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 98/200] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:36 2022] GET /identity/v3/users/7d91e07c08224d54af44714a692b4973 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:36.627910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dc759-b145-4922-b9ec-8b062c409ed0 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:36.668095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 103/201] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:36.669327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da998ec0-0fa9-4d09-9846-149dad78c03c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:36.741537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dc759-b145-4922-b9ec-8b062c409ed0 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:36.777938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da998ec0-0fa9-4d09-9846-149dad78c03c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:36.781332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095a896-19b3-4dee-a8e2-b9e917ad69cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:36.781757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095a896-19b3-4dee-a8e2-b9e917ad69cf None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:36.782109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095a896-19b3-4dee-a8e2-b9e917ad69cf None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:36.782453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095a896-19b3-4dee-a8e2-b9e917ad69cf None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:36.783729 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095a896-19b3-4dee-a8e2-b9e917ad69cf None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:36.785275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095a896-19b3-4dee-a8e2-b9e917ad69cf None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:36.787613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 99/202] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:36 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:36.788542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095a896-19b3-4dee-a8e2-b9e917ad69cf None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:36.818236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a91f63-83ee-4caa-8f02-03e803fb9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:36.819228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a91f63-83ee-4caa-8f02-03e803fb9298 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:36.819778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a91f63-83ee-4caa-8f02-03e803fb9298 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:36.820202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a91f63-83ee-4caa-8f02-03e803fb9298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:36.832544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:36.851747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:36.906920 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a91f63-83ee-4caa-8f02-03e803fb9298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:36.909959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a91f63-83ee-4caa-8f02-03e803fb9298 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:36.914205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095a896-19b3-4dee-a8e2-b9e917ad69cf None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:36.964854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b3d6b-b4d2-466e-8474-7b89bd9c9bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:36.966361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b3d6b-b4d2-466e-8474-7b89bd9c9bbc None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:36.966805 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b3d6b-b4d2-466e-8474-7b89bd9c9bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:36.967241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b3d6b-b4d2-466e-8474-7b89bd9c9bbc None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:36.969245 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b3d6b-b4d2-466e-8474-7b89bd9c9bbc None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:36.971393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b3d6b-b4d2-466e-8474-7b89bd9c9bbc None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:36.979435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 100/203] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:36 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:36.979435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b3d6b-b4d2-466e-8474-7b89bd9c9bbc None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:37.025517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 104/204] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:37.030955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a91f63-83ee-4caa-8f02-03e803fb9298 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:37.091553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:37.113300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b3d6b-b4d2-466e-8474-7b89bd9c9bbc None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:37.123496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:37.139138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a91f63-83ee-4caa-8f02-03e803fb9298 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:37.162268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4914d-3002-4581-8394-d001660a0108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:37.163402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4914d-3002-4581-8394-d001660a0108 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:37.164236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4914d-3002-4581-8394-d001660a0108 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:37.164686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4914d-3002-4581-8394-d001660a0108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:37.165989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4914d-3002-4581-8394-d001660a0108 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e, user.id=16222bf7be05422e98ecbd86571482e9) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:37.166482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4914d-3002-4581-8394-d001660a0108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:37.168942 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4914d-3002-4581-8394-d001660a0108 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:37.175466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b33e26-03be-48c8-a109-8da181d47831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:37.176243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b33e26-03be-48c8-a109-8da181d47831 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:37.176688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 101/205] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:36 2022] GET /identity/v3/role_assignments?user.id=16222bf7be05422e98ecbd86571482e9&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:37.176976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b33e26-03be-48c8-a109-8da181d47831 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:37.177361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b33e26-03be-48c8-a109-8da181d47831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:37.178114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4914d-3002-4581-8394-d001660a0108 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:37.178402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b33e26-03be-48c8-a109-8da181d47831 None admin] RBAC: Adding query filter params (role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e, user.id=16222bf7be05422e98ecbd86571482e9) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:37.178819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b33e26-03be-48c8-a109-8da181d47831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:37.181059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b33e26-03be-48c8-a109-8da181d47831 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:37.196904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 105/206] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:37 2022] GET /identity/v3/role_assignments?user.id=16222bf7be05422e98ecbd86571482e9&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:37.346789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4914d-3002-4581-8394-d001660a0108 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:37.404687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752b285-204d-4714-b8b3-7da8f5014e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:37.405616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752b285-204d-4714-b8b3-7da8f5014e5c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:37.405898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752b285-204d-4714-b8b3-7da8f5014e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:37.406176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752b285-204d-4714-b8b3-7da8f5014e5c None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:37.407428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752b285-204d-4714-b8b3-7da8f5014e5c None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:37.408930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752b285-204d-4714-b8b3-7da8f5014e5c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:37.410714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 102/207] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:37 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:37.417534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b33e26-03be-48c8-a109-8da181d47831 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:37.542071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b33e26-03be-48c8-a109-8da181d47831 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:37.574170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:37.590639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326e33e-e26e-446b-a954-dc4031d19c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:37.591619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326e33e-e26e-446b-a954-dc4031d19c56 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:37.592103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326e33e-e26e-446b-a954-dc4031d19c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:37.592594 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326e33e-e26e-446b-a954-dc4031d19c56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:37.593658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326e33e-e26e-446b-a954-dc4031d19c56 None admin] RBAC: Adding query filter params (role.id=2acd8a15a63a4261b1ae12f09d28722f, scope.project.id=84b308b5ae9a4e0482f37715f327cd20, user.id=7d91e07c08224d54af44714a692b4973) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:37.594171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326e33e-e26e-446b-a954-dc4031d19c56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:37.595947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326e33e-e26e-446b-a954-dc4031d19c56 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:37.610922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 106/208] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:37 2022] GET /identity/v3/role_assignments?user.id=7d91e07c08224d54af44714a692b4973&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=2acd8a15a63a4261b1ae12f09d28722f => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:38.396989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec37e89-0e78-4e15-9217-26dc54d3a8ec None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:38.397495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec37e89-0e78-4e15-9217-26dc54d3a8ec None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:38.397569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec37e89-0e78-4e15-9217-26dc54d3a8ec None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:38.399814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 103/209] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:38.405251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b325c-6d09-42c8-b4a7-867d9f1c26ee None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:38.405534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b325c-6d09-42c8-b4a7-867d9f1c26ee None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:38.405808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b325c-6d09-42c8-b4a7-867d9f1c26ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:38.419306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b001-e5f2-4a15-ab86-8690e128ce9c None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:38.419581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b001-e5f2-4a15-ab86-8690e128ce9c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:38.419856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b001-e5f2-4a15-ab86-8690e128ce9c None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:38.421384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 104/210] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:15:38.426092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4440d56-afe1-471b-aa29-36e813ed1dbf None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:38.426377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4440d56-afe1-471b-aa29-36e813ed1dbf None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:38.426648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4440d56-afe1-471b-aa29-36e813ed1dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:38.427354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-273b325c-6d09-42c8-b4a7-867d9f1c26ee None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:38.428985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-273b325c-6d09-42c8-b4a7-867d9f1c26ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:38.443493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a4440d56-afe1-471b-aa29-36e813ed1dbf None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:38.444668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a4440d56-afe1-471b-aa29-36e813ed1dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:38.475776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:38.507809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 107/211] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:38.527055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 105/212] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:38.533957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b325c-6d09-42c8-b4a7-867d9f1c26ee None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:38.553920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4440d56-afe1-471b-aa29-36e813ed1dbf None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:38.664857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b325c-6d09-42c8-b4a7-867d9f1c26ee None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:38.667450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4440d56-afe1-471b-aa29-36e813ed1dbf None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:38.685490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:38.707344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85e48f-724c-4f4e-9a7a-7b27ba92947f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:38.708156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85e48f-724c-4f4e-9a7a-7b27ba92947f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:38.708441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85e48f-724c-4f4e-9a7a-7b27ba92947f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:38.708720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85e48f-724c-4f4e-9a7a-7b27ba92947f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:38.709958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85e48f-724c-4f4e-9a7a-7b27ba92947f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:38.712057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85e48f-724c-4f4e-9a7a-7b27ba92947f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:38.713750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 108/213] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:38.719366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85e48f-724c-4f4e-9a7a-7b27ba92947f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:38.719872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feb2bb-221c-463e-9e45-92e28ded0383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:38.720992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feb2bb-221c-463e-9e45-92e28ded0383 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:38.721474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feb2bb-221c-463e-9e45-92e28ded0383 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:38.722312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feb2bb-221c-463e-9e45-92e28ded0383 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:38.737534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83feb2bb-221c-463e-9e45-92e28ded0383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-83feb2bb-221c-463e-9e45-92e28ded0383 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:38.749752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:38.750625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:38.751386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:38.759542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 106/214] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:15:38.761141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feb2bb-221c-463e-9e45-92e28ded0383 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:38.791052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:38.850220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85e48f-724c-4f4e-9a7a-7b27ba92947f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:38.876712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feb2bb-221c-463e-9e45-92e28ded0383 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:38.903852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd7ff7-6ad4-491c-bbd2-c103166c91e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:38.905117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd7ff7-6ad4-491c-bbd2-c103166c91e7 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:38.905619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd7ff7-6ad4-491c-bbd2-c103166c91e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:38.906239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd7ff7-6ad4-491c-bbd2-c103166c91e7 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:38.907724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd7ff7-6ad4-491c-bbd2-c103166c91e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:38.909440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd7ff7-6ad4-491c-bbd2-c103166c91e7 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:38.911300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210353d0-d0f5-4851-a46a-15f6b0ecc51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:38.911962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210353d0-d0f5-4851-a46a-15f6b0ecc51f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:38.912242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210353d0-d0f5-4851-a46a-15f6b0ecc51f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:38.912516 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210353d0-d0f5-4851-a46a-15f6b0ecc51f None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:38.913349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210353d0-d0f5-4851-a46a-15f6b0ecc51f None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:38.913652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210353d0-d0f5-4851-a46a-15f6b0ecc51f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:38.915809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210353d0-d0f5-4851-a46a-15f6b0ecc51f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:38.924151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 107/215] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:38.939496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210353d0-d0f5-4851-a46a-15f6b0ecc51f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:38.945576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 109/216] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:15:38 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:15:39.048331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:39.059227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210353d0-d0f5-4851-a46a-15f6b0ecc51f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:39.096403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819005e-4a8a-42c8-943f-b0abac4e279c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:39.097688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819005e-4a8a-42c8-943f-b0abac4e279c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:39.098116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819005e-4a8a-42c8-943f-b0abac4e279c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:39.098535 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819005e-4a8a-42c8-943f-b0abac4e279c None admin] PATH_INFO: `/v3/users/31c9427ce622494f875b977076a7a3e2` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:39.114651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819005e-4a8a-42c8-943f-b0abac4e279c None admin] RBAC: Authorizing `identity:get_user(user_id=31c9427ce622494f875b977076a7a3e2)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:39.116390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819005e-4a8a-42c8-943f-b0abac4e279c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:39.118507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 108/217] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:38 2022] GET /identity/v3/users/31c9427ce622494f875b977076a7a3e2 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:39.130477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd7ff7-6ad4-491c-bbd2-c103166c91e7 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:39.180030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:39.247694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd7ff7-6ad4-491c-bbd2-c103166c91e7 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:39.284712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd27e9d-5330-4afb-b3c7-aaa90ef8c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:39.285443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd27e9d-5330-4afb-b3c7-aaa90ef8c8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:39.285723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd27e9d-5330-4afb-b3c7-aaa90ef8c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:39.286022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd27e9d-5330-4afb-b3c7-aaa90ef8c8f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:39.362981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd27e9d-5330-4afb-b3c7-aaa90ef8c8f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:39.364479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd27e9d-5330-4afb-b3c7-aaa90ef8c8f3 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:39.407151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:39.472373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 110/218] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:15:39.478027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819005e-4a8a-42c8-943f-b0abac4e279c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:39.605409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819005e-4a8a-42c8-943f-b0abac4e279c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:39.659798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5401d-138a-40bd-88ae-3d9fb768291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:39.660706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5401d-138a-40bd-88ae-3d9fb768291b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:39.661012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5401d-138a-40bd-88ae-3d9fb768291b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:39.661285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5401d-138a-40bd-88ae-3d9fb768291b None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:39.662605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5401d-138a-40bd-88ae-3d9fb768291b None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:39.664112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5401d-138a-40bd-88ae-3d9fb768291b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:39.666885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 109/219] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:39 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:39.672937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd27e9d-5330-4afb-b3c7-aaa90ef8c8f3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:39.810037 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd27e9d-5330-4afb-b3c7-aaa90ef8c8f3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:39.855798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a5203-1e3e-4f92-9a9e-84236dfed7a3 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:39.856080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a5203-1e3e-4f92-9a9e-84236dfed7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:39.856545 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a5203-1e3e-4f92-9a9e-84236dfed7a3 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:39.858331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 110/220] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:39.862792 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1447d-0c57-41d0-8c77-0e84d9c8367a None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:39.863306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1447d-0c57-41d0-8c77-0e84d9c8367a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:39.863963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1447d-0c57-41d0-8c77-0e84d9c8367a None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:39.865119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc7b0c-ca8c-4b51-8a34-9a05feb2d494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:39.865777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 111/221] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:15:39.865979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc7b0c-ca8c-4b51-8a34-9a05feb2d494 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:39.866412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc7b0c-ca8c-4b51-8a34-9a05feb2d494 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:39.866822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc7b0c-ca8c-4b51-8a34-9a05feb2d494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:39.867338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d319153-2dfe-4bfb-a72a-c75c9f5ae78c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:39.868069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc7b0c-ca8c-4b51-8a34-9a05feb2d494 None admin] RBAC: Adding query filter params (role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e, user.id=31c9427ce622494f875b977076a7a3e2) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:39.868633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc7b0c-ca8c-4b51-8a34-9a05feb2d494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:39.870538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc7b0c-ca8c-4b51-8a34-9a05feb2d494 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:39.871937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d319153-2dfe-4bfb-a72a-c75c9f5ae78c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:39.872329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d319153-2dfe-4bfb-a72a-c75c9f5ae78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:39.876958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 111/222] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:39 2022] GET /identity/v3/role_assignments?user.id=31c9427ce622494f875b977076a7a3e2&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=8de5789832e84f9388b1295d479fca6e => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:39.878623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96809707-f029-4be1-8371-ceabbd25479e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:39.879076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96809707-f029-4be1-8371-ceabbd25479e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:39.880370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96809707-f029-4be1-8371-ceabbd25479e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:39.888536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:39.893470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1d319153-2dfe-4bfb-a72a-c75c9f5ae78c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:39.894809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1d319153-2dfe-4bfb-a72a-c75c9f5ae78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:39.903228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-96809707-f029-4be1-8371-ceabbd25479e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:39.904750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-96809707-f029-4be1-8371-ceabbd25479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:39.925413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:40.047501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 112/223] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:40.056087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 112/224] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:40.070549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d319153-2dfe-4bfb-a72a-c75c9f5ae78c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:40.104131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96809707-f029-4be1-8371-ceabbd25479e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:40.173624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d319153-2dfe-4bfb-a72a-c75c9f5ae78c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:40.209330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b7dd-1c47-4cc9-bf1c-c1ec13190247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:40.210415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b7dd-1c47-4cc9-bf1c-c1ec13190247 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:40.210869 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b7dd-1c47-4cc9-bf1c-c1ec13190247 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:40.211470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b7dd-1c47-4cc9-bf1c-c1ec13190247 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:40.222275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21b7dd-1c47-4cc9-bf1c-c1ec13190247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-ad21b7dd-1c47-4cc9-bf1c-c1ec13190247 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:40.230435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:40.231337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:40.232104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:40.234497 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 113/225] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:40.240308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b7dd-1c47-4cc9-bf1c-c1ec13190247 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:40.257267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:40.356706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b7dd-1c47-4cc9-bf1c-c1ec13190247 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:40.374397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96809707-f029-4be1-8371-ceabbd25479e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:40.394427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889d4fd-55a0-4277-ab88-d3c62d85dfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:40.395972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889d4fd-55a0-4277-ab88-d3c62d85dfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:40.396450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889d4fd-55a0-4277-ab88-d3c62d85dfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:40.396994 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889d4fd-55a0-4277-ab88-d3c62d85dfa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:40.398100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889d4fd-55a0-4277-ab88-d3c62d85dfa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:40.399465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889d4fd-55a0-4277-ab88-d3c62d85dfa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:40.401661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889d4fd-55a0-4277-ab88-d3c62d85dfa5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:40.412878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 114/226] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:40.423060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889d4fd-55a0-4277-ab88-d3c62d85dfa5 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:40.439589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5a772-666f-433e-a961-65c628918b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:40.440413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5a772-666f-433e-a961-65c628918b53 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:40.440695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5a772-666f-433e-a961-65c628918b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:40.441005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5a772-666f-433e-a961-65c628918b53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:40.446477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:40.462238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5a772-666f-433e-a961-65c628918b53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-67c5a772-666f-433e-a961-65c628918b53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:40.478606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:40.479481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:40.480239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:40.480812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 113/227] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:15:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:40.488945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5a772-666f-433e-a961-65c628918b53 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:40.554025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:40.618348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889d4fd-55a0-4277-ab88-d3c62d85dfa5 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:40.656099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291c6cb-6908-4ad5-b803-ca2dc0c00bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:40.657466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291c6cb-6908-4ad5-b803-ca2dc0c00bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:40.657964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291c6cb-6908-4ad5-b803-ca2dc0c00bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:40.658416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291c6cb-6908-4ad5-b803-ca2dc0c00bb0 None admin] PATH_INFO: `/v3/users/16222bf7be05422e98ecbd86571482e9` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:40.659798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291c6cb-6908-4ad5-b803-ca2dc0c00bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=16222bf7be05422e98ecbd86571482e9)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:40.661488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291c6cb-6908-4ad5-b803-ca2dc0c00bb0 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:40.663610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 115/228] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:40 2022] GET /identity/v3/users/16222bf7be05422e98ecbd86571482e9 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:40.669953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291c6cb-6908-4ad5-b803-ca2dc0c00bb0 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:40.676396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5a772-666f-433e-a961-65c628918b53 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:40.781598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151eacd-fcfa-4f18-85e7-4a2412cb3680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:40.782275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151eacd-fcfa-4f18-85e7-4a2412cb3680 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:40.782710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151eacd-fcfa-4f18-85e7-4a2412cb3680 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:40.783150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151eacd-fcfa-4f18-85e7-4a2412cb3680 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:40.784059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151eacd-fcfa-4f18-85e7-4a2412cb3680 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:40.784502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151eacd-fcfa-4f18-85e7-4a2412cb3680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:40.786137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151eacd-fcfa-4f18-85e7-4a2412cb3680 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:40.804017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 114/229] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:15:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:40.810500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291c6cb-6908-4ad5-b803-ca2dc0c00bb0 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:40.819609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151eacd-fcfa-4f18-85e7-4a2412cb3680 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:40.849532 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0a9e7-2280-4984-9097-61545d352cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:40.850870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0a9e7-2280-4984-9097-61545d352cab None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:40.851330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0a9e7-2280-4984-9097-61545d352cab None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:40.851807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0a9e7-2280-4984-9097-61545d352cab None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:40.853827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0a9e7-2280-4984-9097-61545d352cab None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:40.855453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0a9e7-2280-4984-9097-61545d352cab None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:40.857756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 116/230] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:40 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:40.863592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0a9e7-2280-4984-9097-61545d352cab None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:40.867226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:40.967857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0a9e7-2280-4984-9097-61545d352cab None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:40.989863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:41.003803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d0dae-1b56-4e07-958c-b89b9061eee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:41.004907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d0dae-1b56-4e07-958c-b89b9061eee8 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:41.005337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d0dae-1b56-4e07-958c-b89b9061eee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:41.005785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d0dae-1b56-4e07-958c-b89b9061eee8 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/users/16222bf7be05422e98ecbd86571482e9/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:41.007886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d0dae-1b56-4e07-958c-b89b9061eee8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cbcf4de61f4fb59bbc787e33742b7e, user_id=16222bf7be05422e98ecbd86571482e9, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:41.009547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d0dae-1b56-4e07-958c-b89b9061eee8 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:41.024998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 117/231] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:15:40 2022] PUT /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/users/16222bf7be05422e98ecbd86571482e9/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:41.091026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151eacd-fcfa-4f18-85e7-4a2412cb3680 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:41.144617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1765d-afe8-46fd-a079-c4475017d1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:41.145461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1765d-afe8-46fd-a079-c4475017d1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:41.145739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1765d-afe8-46fd-a079-c4475017d1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:41.146010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1765d-afe8-46fd-a079-c4475017d1f0 None admin] PATH_INFO: `/v3/users/16222bf7be05422e98ecbd86571482e9` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:41.147371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1765d-afe8-46fd-a079-c4475017d1f0 None admin] RBAC: Authorizing `identity:get_user(user_id=16222bf7be05422e98ecbd86571482e9)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:41.148970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1765d-afe8-46fd-a079-c4475017d1f0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:41.151554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 115/232] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:40 2022] GET /identity/v3/users/16222bf7be05422e98ecbd86571482e9 => generated 283 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:41.156795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d0dae-1b56-4e07-958c-b89b9061eee8 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:41.249317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:41.281607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d0dae-1b56-4e07-958c-b89b9061eee8 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:41.319300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852200-3a06-48fb-9351-032db485cc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:41.320541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852200-3a06-48fb-9351-032db485cc02 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:41.321126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852200-3a06-48fb-9351-032db485cc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:41.321602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852200-3a06-48fb-9351-032db485cc02 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:41.323034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab852200-3a06-48fb-9351-032db485cc02 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:41.324804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab852200-3a06-48fb-9351-032db485cc02 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:41.326803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 118/233] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:41 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:41.333065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1765d-afe8-46fd-a079-c4475017d1f0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:41.454849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1765d-afe8-46fd-a079-c4475017d1f0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:41.485720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:41.498501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f6739-43ff-4b4d-8ba5-07ccdf6807b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:41.499765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f6739-43ff-4b4d-8ba5-07ccdf6807b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:41.500249 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f6739-43ff-4b4d-8ba5-07ccdf6807b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:41.500728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f6739-43ff-4b4d-8ba5-07ccdf6807b8 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/users/16222bf7be05422e98ecbd86571482e9/roles/e2a3f2f08d18410a954a4d6b8314562d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:41.503204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f6739-43ff-4b4d-8ba5-07ccdf6807b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cbcf4de61f4fb59bbc787e33742b7e, user_id=16222bf7be05422e98ecbd86571482e9, role_id=e2a3f2f08d18410a954a4d6b8314562d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:41.504917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f6739-43ff-4b4d-8ba5-07ccdf6807b8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:41.532712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 116/234] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:15:41 2022] PUT /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/users/16222bf7be05422e98ecbd86571482e9/roles/e2a3f2f08d18410a954a4d6b8314562d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:41.892068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d58c09-0bae-4064-b08f-de7a03bb3b4a None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:41.893129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d58c09-0bae-4064-b08f-de7a03bb3b4a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:41.893644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d58c09-0bae-4064-b08f-de7a03bb3b4a None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:41.895329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 119/235] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:41.902071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c69c9e-9133-4a64-82bd-faf84d41d8f3 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:41.903020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c69c9e-9133-4a64-82bd-faf84d41d8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:41.903522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c69c9e-9133-4a64-82bd-faf84d41d8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:41.921645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-f5c69c9e-9133-4a64-82bd-faf84d41d8f3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:41.923157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f5c69c9e-9133-4a64-82bd-faf84d41d8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:41.996981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 117/236] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:42.023329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d58c09-0bae-4064-b08f-de7a03bb3b4a None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:42.121823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869afff-200b-4195-95fb-fe77d9b0bc7d None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:42.122169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869afff-200b-4195-95fb-fe77d9b0bc7d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:42.122374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869afff-200b-4195-95fb-fe77d9b0bc7d None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:42.123767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 118/237] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:42.128985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53eed35-a770-4bc8-aa08-6668136bd46c None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:42.129547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53eed35-a770-4bc8-aa08-6668136bd46c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:42.130024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53eed35-a770-4bc8-aa08-6668136bd46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:42.137664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d58c09-0bae-4064-b08f-de7a03bb3b4a None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:42.168045 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e53eed35-a770-4bc8-aa08-6668136bd46c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:42.169202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e53eed35-a770-4bc8-aa08-6668136bd46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:42.178510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:42.189247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea671a73-3cd1-4d32-953e-24402942b536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:42.190213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea671a73-3cd1-4d32-953e-24402942b536 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:42.190503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea671a73-3cd1-4d32-953e-24402942b536 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:42.190783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea671a73-3cd1-4d32-953e-24402942b536 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:42.210130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea671a73-3cd1-4d32-953e-24402942b536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-ea671a73-3cd1-4d32-953e-24402942b536 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:42.215072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:42.215911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:42.216677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 120/238] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:42.222363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea671a73-3cd1-4d32-953e-24402942b536 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:42.334197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea671a73-3cd1-4d32-953e-24402942b536 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:42.339986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 119/239] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:42.370463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53eed35-a770-4bc8-aa08-6668136bd46c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:42.396478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3389194-53f0-4453-89cc-91d0211cde44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:42.397182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3389194-53f0-4453-89cc-91d0211cde44 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:42.397464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3389194-53f0-4453-89cc-91d0211cde44 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:42.397738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3389194-53f0-4453-89cc-91d0211cde44 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:42.398491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3389194-53f0-4453-89cc-91d0211cde44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:42.399368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3389194-53f0-4453-89cc-91d0211cde44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:42.400837 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3389194-53f0-4453-89cc-91d0211cde44 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:42.420432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 121/240] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:42.424977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3389194-53f0-4453-89cc-91d0211cde44 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:42.452279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:42.504428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:42.560616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53eed35-a770-4bc8-aa08-6668136bd46c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:42.582267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3389194-53f0-4453-89cc-91d0211cde44 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:42.611562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eff9a3-3ae6-4351-9f24-53c9fd30fb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:42.613103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eff9a3-3ae6-4351-9f24-53c9fd30fb72 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:42.613596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eff9a3-3ae6-4351-9f24-53c9fd30fb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:42.614362 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eff9a3-3ae6-4351-9f24-53c9fd30fb72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:42.615915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eff9a3-3ae6-4351-9f24-53c9fd30fb72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:42.619102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eff9a3-3ae6-4351-9f24-53c9fd30fb72 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:42.622328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 120/241] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:42.623768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2597fd-2b5f-45c1-9229-03a688b68c45 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:42.624253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2597fd-2b5f-45c1-9229-03a688b68c45 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:42.624726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2597fd-2b5f-45c1-9229-03a688b68c45 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:42.626193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 121/242] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:42.631231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2597fd-2b5f-45c1-9229-03a688b68c45 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:42.637824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e283ea6-9d5e-4137-9aa9-29c61f21a511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:42.638723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e283ea6-9d5e-4137-9aa9-29c61f21a511 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:42.639002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e283ea6-9d5e-4137-9aa9-29c61f21a511 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:42.639275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e283ea6-9d5e-4137-9aa9-29c61f21a511 None admin] PATH_INFO: `/v3/users/4f1426dbd23141659191f6f1c83ed794` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:42.657618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e283ea6-9d5e-4137-9aa9-29c61f21a511 None admin] RBAC: Authorizing `identity:get_user(user_id=4f1426dbd23141659191f6f1c83ed794)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:42.659399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e283ea6-9d5e-4137-9aa9-29c61f21a511 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:42.661190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 122/243] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:42 2022] GET /identity/v3/users/4f1426dbd23141659191f6f1c83ed794 => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:42.664952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bb992-eee6-4bae-90ec-ba2fd1bcc924 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:42.664952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bb992-eee6-4bae-90ec-ba2fd1bcc924 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:42.664952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bb992-eee6-4bae-90ec-ba2fd1bcc924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:42.690786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-e10bb992-eee6-4bae-90ec-ba2fd1bcc924 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:42.692190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e10bb992-eee6-4bae-90ec-ba2fd1bcc924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:42.754152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2597fd-2b5f-45c1-9229-03a688b68c45 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:42.765845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 123/244] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:42.769469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:42.773860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bb992-eee6-4bae-90ec-ba2fd1bcc924 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:42.804491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46496158-f1e8-4340-8ee2-30184671a965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:42.805409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46496158-f1e8-4340-8ee2-30184671a965 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:42.805694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46496158-f1e8-4340-8ee2-30184671a965 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:42.805970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46496158-f1e8-4340-8ee2-30184671a965 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:42.806842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46496158-f1e8-4340-8ee2-30184671a965 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:42.808779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46496158-f1e8-4340-8ee2-30184671a965 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:42.831632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:42.840894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 122/245] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:15:42 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:42.840894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46496158-f1e8-4340-8ee2-30184671a965 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:42.944317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bb992-eee6-4bae-90ec-ba2fd1bcc924 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:42.962733 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46496158-f1e8-4340-8ee2-30184671a965 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:43.006843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a208c-bf4c-424e-b906-8ecc7d47172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:43.008031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a208c-bf4c-424e-b906-8ecc7d47172e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.008979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a208c-bf4c-424e-b906-8ecc7d47172e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.009262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a208c-bf4c-424e-b906-8ecc7d47172e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.021821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a208c-bf4c-424e-b906-8ecc7d47172e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:43.022850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41a481-4572-4a0d-825b-63f504cbe579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:43.023810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41a481-4572-4a0d-825b-63f504cbe579 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.025010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41a481-4572-4a0d-825b-63f504cbe579 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.025499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41a481-4572-4a0d-825b-63f504cbe579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-0b5a208c-bf4c-424e-b906-8ecc7d47172e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:43.030596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:43.031485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:43.032331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 123/246] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:43.038866 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a208c-bf4c-424e-b906-8ecc7d47172e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:43.078756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:43.162975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a208c-bf4c-424e-b906-8ecc7d47172e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:43.195230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41a481-4572-4a0d-825b-63f504cbe579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:43.197168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41a481-4572-4a0d-825b-63f504cbe579 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:43.214741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965d0c6-08a5-4dfa-819c-adc75debd8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:43.215734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965d0c6-08a5-4dfa-819c-adc75debd8ad None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.216016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965d0c6-08a5-4dfa-819c-adc75debd8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.216760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965d0c6-08a5-4dfa-819c-adc75debd8ad None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.217575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965d0c6-08a5-4dfa-819c-adc75debd8ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:43.217888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965d0c6-08a5-4dfa-819c-adc75debd8ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:43.219566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965d0c6-08a5-4dfa-819c-adc75debd8ad None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:43.231860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 124/247] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:43.238241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965d0c6-08a5-4dfa-819c-adc75debd8ad None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:43.262187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:43.334466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 124/248] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:43.341209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41a481-4572-4a0d-825b-63f504cbe579 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:43.357792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:43.369486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965d0c6-08a5-4dfa-819c-adc75debd8ad None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:43.405710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f011e5-b8c2-4a62-99f6-4a2908a2ae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:43.406947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f011e5-b8c2-4a62-99f6-4a2908a2ae91 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.407451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f011e5-b8c2-4a62-99f6-4a2908a2ae91 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.407948 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f011e5-b8c2-4a62-99f6-4a2908a2ae91 None admin] PATH_INFO: `/v3/users/31c9427ce622494f875b977076a7a3e2` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.409444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f011e5-b8c2-4a62-99f6-4a2908a2ae91 None admin] RBAC: Authorizing `identity:get_user(user_id=31c9427ce622494f875b977076a7a3e2)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:43.411107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f011e5-b8c2-4a62-99f6-4a2908a2ae91 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:43.413004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 125/249] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:43 2022] GET /identity/v3/users/31c9427ce622494f875b977076a7a3e2 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:43.419327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f011e5-b8c2-4a62-99f6-4a2908a2ae91 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:43.458723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41a481-4572-4a0d-825b-63f504cbe579 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:43.499204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56655-ca47-46bf-a9a9-09469b9f21c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:43.500229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56655-ca47-46bf-a9a9-09469b9f21c5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.500568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56655-ca47-46bf-a9a9-09469b9f21c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.500844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56655-ca47-46bf-a9a9-09469b9f21c5 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.502079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db56655-ca47-46bf-a9a9-09469b9f21c5 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:43.503849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db56655-ca47-46bf-a9a9-09469b9f21c5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:43.505437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 125/250] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:43 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:43.512725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56655-ca47-46bf-a9a9-09469b9f21c5 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:43.531597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f011e5-b8c2-4a62-99f6-4a2908a2ae91 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:43.549293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:43.569430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380aac32-f37c-4922-bdaa-49bc01fc98f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:43.571311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380aac32-f37c-4922-bdaa-49bc01fc98f4 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.571933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380aac32-f37c-4922-bdaa-49bc01fc98f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.572410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380aac32-f37c-4922-bdaa-49bc01fc98f4 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.574977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380aac32-f37c-4922-bdaa-49bc01fc98f4 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:43.576657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380aac32-f37c-4922-bdaa-49bc01fc98f4 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:43.578904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 126/251] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:43 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:43.589049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380aac32-f37c-4922-bdaa-49bc01fc98f4 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:43.626722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:43.631409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56655-ca47-46bf-a9a9-09469b9f21c5 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:43.676134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993ef1b-d745-4012-96e7-ca316b48e289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:43.676801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993ef1b-d745-4012-96e7-ca316b48e289 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.677106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993ef1b-d745-4012-96e7-ca316b48e289 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.677431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993ef1b-d745-4012-96e7-ca316b48e289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.678324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993ef1b-d745-4012-96e7-ca316b48e289 None admin] RBAC: Adding query filter params (role.id=2acd8a15a63a4261b1ae12f09d28722f, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e, user.id=4f1426dbd23141659191f6f1c83ed794) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:43.678634 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993ef1b-d745-4012-96e7-ca316b48e289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:43.680668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993ef1b-d745-4012-96e7-ca316b48e289 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:43.687937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 126/252] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:43 2022] GET /identity/v3/role_assignments?user.id=4f1426dbd23141659191f6f1c83ed794&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=2acd8a15a63a4261b1ae12f09d28722f => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:43.720173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380aac32-f37c-4922-bdaa-49bc01fc98f4 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:43.762179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0b47e-2686-4aff-bdbd-531003c1be86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:43.763102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0b47e-2686-4aff-bdbd-531003c1be86 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.763383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0b47e-2686-4aff-bdbd-531003c1be86 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.763661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0b47e-2686-4aff-bdbd-531003c1be86 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/users/31c9427ce622494f875b977076a7a3e2/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.765838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0b47e-2686-4aff-bdbd-531003c1be86 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cbcf4de61f4fb59bbc787e33742b7e, user_id=31c9427ce622494f875b977076a7a3e2, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:43.767342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0b47e-2686-4aff-bdbd-531003c1be86 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:43.783966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 127/253] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:15:43 2022] PUT /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/users/31c9427ce622494f875b977076a7a3e2/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:43.863461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bc59f-6a41-4c2f-b8e1-0e601972f755 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.863814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bc59f-6a41-4c2f-b8e1-0e601972f755 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.864018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bc59f-6a41-4c2f-b8e1-0e601972f755 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.865453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 127/254] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:43.871991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee750b8-c6ac-4b43-9c84-9fc82ab4e73b None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:43.872293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee750b8-c6ac-4b43-9c84-9fc82ab4e73b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:43.872560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee750b8-c6ac-4b43-9c84-9fc82ab4e73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:43.894896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-cee750b8-c6ac-4b43-9c84-9fc82ab4e73b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:43.896239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-cee750b8-c6ac-4b43-9c84-9fc82ab4e73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:43.979012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 128/255] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:44.021400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bc59f-6a41-4c2f-b8e1-0e601972f755 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:44.117031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:44.147366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bc59f-6a41-4c2f-b8e1-0e601972f755 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:44.189368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27177cf0-a28c-4a3f-921a-af318fcbe45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:44.190174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27177cf0-a28c-4a3f-921a-af318fcbe45a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:44.190453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27177cf0-a28c-4a3f-921a-af318fcbe45a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:44.190721 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27177cf0-a28c-4a3f-921a-af318fcbe45a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:44.202941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27177cf0-a28c-4a3f-921a-af318fcbe45a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-27177cf0-a28c-4a3f-921a-af318fcbe45a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:44.210765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:44.212198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:44.213551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 128/256] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:44.220304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee750b8-c6ac-4b43-9c84-9fc82ab4e73b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:44.244243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:44.353618 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee750b8-c6ac-4b43-9c84-9fc82ab4e73b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:44.389900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c14d27-3cd4-4265-b675-e43a4078d470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:44.390575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c14d27-3cd4-4265-b675-e43a4078d470 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:44.390859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c14d27-3cd4-4265-b675-e43a4078d470 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:44.391134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c14d27-3cd4-4265-b675-e43a4078d470 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:44.391899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c14d27-3cd4-4265-b675-e43a4078d470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:44.392203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c14d27-3cd4-4265-b675-e43a4078d470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:44.393653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c14d27-3cd4-4265-b675-e43a4078d470 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:44.403572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 129/257] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:44.408584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1f521-0dc8-4bfe-8779-8c366a070fc8 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:44.408884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1f521-0dc8-4bfe-8779-8c366a070fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:44.409185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1f521-0dc8-4bfe-8779-8c366a070fc8 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:44.409815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c14d27-3cd4-4265-b675-e43a4078d470 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:44.410711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 129/258] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:44.416300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b68b5-8c2f-4006-8b18-068bac8ec19d None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:44.416583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b68b5-8c2f-4006-8b18-068bac8ec19d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:44.416862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b68b5-8c2f-4006-8b18-068bac8ec19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:44.461112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f60b68b5-8c2f-4006-8b18-068bac8ec19d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:44.463214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f60b68b5-8c2f-4006-8b18-068bac8ec19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:44.497391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:44.520392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c14d27-3cd4-4265-b675-e43a4078d470 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:44.580512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb6233-63c9-46e9-95ef-597203ba75c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:44.581430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb6233-63c9-46e9-95ef-597203ba75c3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:44.581706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb6233-63c9-46e9-95ef-597203ba75c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:44.581974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb6233-63c9-46e9-95ef-597203ba75c3 None admin] PATH_INFO: `/v3/users/16222bf7be05422e98ecbd86571482e9` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:44.590970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb6233-63c9-46e9-95ef-597203ba75c3 None admin] RBAC: Authorizing `identity:get_user(user_id=16222bf7be05422e98ecbd86571482e9)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:44.592452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb6233-63c9-46e9-95ef-597203ba75c3 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:44.596289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 130/259] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:44 2022] GET /identity/v3/users/16222bf7be05422e98ecbd86571482e9 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:44.600610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb6233-63c9-46e9-95ef-597203ba75c3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:44.648124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 130/260] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:44.684949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b68b5-8c2f-4006-8b18-068bac8ec19d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:44.760960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb6233-63c9-46e9-95ef-597203ba75c3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:44.799389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:44.823048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a745c-c0a8-4e1c-9282-f098cd43c8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:44.824971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a745c-c0a8-4e1c-9282-f098cd43c8bf None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:44.824971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a745c-c0a8-4e1c-9282-f098cd43c8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:44.824971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a745c-c0a8-4e1c-9282-f098cd43c8bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:44.854235 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:44.875497 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b68b5-8c2f-4006-8b18-068bac8ec19d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:44.938430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cc481-e5b1-496f-ba74-b6345170250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:44.939628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cc481-e5b1-496f-ba74-b6345170250c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:44.940195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cc481-e5b1-496f-ba74-b6345170250c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:44.940676 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cc481-e5b1-496f-ba74-b6345170250c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:44.946838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a745c-c0a8-4e1c-9282-f098cd43c8bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:44.948378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a745c-c0a8-4e1c-9282-f098cd43c8bf None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:44.967490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cc481-e5b1-496f-ba74-b6345170250c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-b47cc481-e5b1-496f-ba74-b6345170250c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:44.976058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:44.976963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:44.978172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:44.987179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 131/261] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:15:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:15:44.993035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cc481-e5b1-496f-ba74-b6345170250c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:45.135671 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 131/262] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:45.141418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a745c-c0a8-4e1c-9282-f098cd43c8bf None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:45.175047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cc481-e5b1-496f-ba74-b6345170250c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:45.212691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:45.233455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a28b4-0ab9-4717-adfb-d3d7e897ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:45.234140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a28b4-0ab9-4717-adfb-d3d7e897ce5c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:45.234425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a28b4-0ab9-4717-adfb-d3d7e897ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:45.234699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a28b4-0ab9-4717-adfb-d3d7e897ce5c None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:45.235458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a28b4-0ab9-4717-adfb-d3d7e897ce5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:45.235771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a28b4-0ab9-4717-adfb-d3d7e897ce5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:45.237169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a28b4-0ab9-4717-adfb-d3d7e897ce5c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:45.253069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 132/263] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:15:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:45.253653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:45.259411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a28b4-0ab9-4717-adfb-d3d7e897ce5c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:45.387633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a745c-c0a8-4e1c-9282-f098cd43c8bf None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:45.453115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1232f-a8be-461f-9dab-55547b89207f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:45.454086 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1232f-a8be-461f-9dab-55547b89207f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:45.454367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1232f-a8be-461f-9dab-55547b89207f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:45.454642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1232f-a8be-461f-9dab-55547b89207f None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:45.456290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a28b4-0ab9-4717-adfb-d3d7e897ce5c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:45.458843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1232f-a8be-461f-9dab-55547b89207f None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:45.460447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1232f-a8be-461f-9dab-55547b89207f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:45.462741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 132/264] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:45 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:45.469003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1232f-a8be-461f-9dab-55547b89207f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:45.492264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865952d-5477-4234-9306-57c8a8ca3d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:45.493727 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865952d-5477-4234-9306-57c8a8ca3d01 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:45.494002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865952d-5477-4234-9306-57c8a8ca3d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:45.494273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865952d-5477-4234-9306-57c8a8ca3d01 None admin] PATH_INFO: `/v3/users/16222bf7be05422e98ecbd86571482e9` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:45.495796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865952d-5477-4234-9306-57c8a8ca3d01 None admin] RBAC: Authorizing `identity:get_user(user_id=16222bf7be05422e98ecbd86571482e9)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:45.497281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865952d-5477-4234-9306-57c8a8ca3d01 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:45.499151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 133/265] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:45 2022] GET /identity/v3/users/16222bf7be05422e98ecbd86571482e9 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:45.504379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865952d-5477-4234-9306-57c8a8ca3d01 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:45.562906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:45.652577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865952d-5477-4234-9306-57c8a8ca3d01 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:45.680040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1232f-a8be-461f-9dab-55547b89207f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:45.699239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059b317-395d-44a3-ae06-63ebbc17ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:45.700163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059b317-395d-44a3-ae06-63ebbc17ff9f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:45.700585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059b317-395d-44a3-ae06-63ebbc17ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:45.701061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059b317-395d-44a3-ae06-63ebbc17ff9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:45.711851 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:45.756382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530782b-51e1-4a76-b996-a0ac0b31e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:45.757075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530782b-51e1-4a76-b996-a0ac0b31e113 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:45.757359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530782b-51e1-4a76-b996-a0ac0b31e113 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:45.757634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530782b-51e1-4a76-b996-a0ac0b31e113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:45.759180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530782b-51e1-4a76-b996-a0ac0b31e113 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e, user.id=16222bf7be05422e98ecbd86571482e9) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:45.759493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530782b-51e1-4a76-b996-a0ac0b31e113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:45.761182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530782b-51e1-4a76-b996-a0ac0b31e113 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:45.774263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059b317-395d-44a3-ae06-63ebbc17ff9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:45.775974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059b317-395d-44a3-ae06-63ebbc17ff9f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:45.777616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 133/266] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:45 2022] GET /identity/v3/role_assignments?user.id=16222bf7be05422e98ecbd86571482e9&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:45.835404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:45.889027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 134/267] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:15:45.890612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530782b-51e1-4a76-b996-a0ac0b31e113 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:46.042046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530782b-51e1-4a76-b996-a0ac0b31e113 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:46.086921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce64479-068d-43ab-8f47-8978739419d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:46.087893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce64479-068d-43ab-8f47-8978739419d1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:46.088175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce64479-068d-43ab-8f47-8978739419d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:46.088448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce64479-068d-43ab-8f47-8978739419d1 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:46.089775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce64479-068d-43ab-8f47-8978739419d1 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:46.091289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce64479-068d-43ab-8f47-8978739419d1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:46.093427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 134/268] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:45 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:46.098921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059b317-395d-44a3-ae06-63ebbc17ff9f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:46.234768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059b317-395d-44a3-ae06-63ebbc17ff9f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:46.270437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240169f-01e0-4ea5-9a39-cff69b56c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:46.271154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240169f-01e0-4ea5-9a39-cff69b56c053 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:46.271443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240169f-01e0-4ea5-9a39-cff69b56c053 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:46.271722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240169f-01e0-4ea5-9a39-cff69b56c053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:46.272560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240169f-01e0-4ea5-9a39-cff69b56c053 None admin] RBAC: Adding query filter params (role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e, user.id=16222bf7be05422e98ecbd86571482e9) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:46.272893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240169f-01e0-4ea5-9a39-cff69b56c053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:46.275176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240169f-01e0-4ea5-9a39-cff69b56c053 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:46.281821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 135/269] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:46 2022] GET /identity/v3/role_assignments?user.id=16222bf7be05422e98ecbd86571482e9&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:46.932850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234df60-96c1-4d79-a13d-fceaa1ff6fb4 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:46.933354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234df60-96c1-4d79-a13d-fceaa1ff6fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:46.933631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234df60-96c1-4d79-a13d-fceaa1ff6fb4 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:46.935332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 135/270] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:46.941153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404ff0a-3d54-4e2b-9827-678ab10bd322 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:46.941525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404ff0a-3d54-4e2b-9827-678ab10bd322 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:46.941950 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404ff0a-3d54-4e2b-9827-678ab10bd322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:46.961556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-2404ff0a-3d54-4e2b-9827-678ab10bd322 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:46.963241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2404ff0a-3d54-4e2b-9827-678ab10bd322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:46.982160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf897079-ffe7-494e-9095-b4ed1a9df3fb None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:46.982478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf897079-ffe7-494e-9095-b4ed1a9df3fb None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:46.982994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf897079-ffe7-494e-9095-b4ed1a9df3fb None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:46.984438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:46.985163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 136/271] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:46.990577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce5ffe-2d74-4aed-b73d-1745b03f93ce None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:46.990861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce5ffe-2d74-4aed-b73d-1745b03f93ce None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:46.991140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce5ffe-2d74-4aed-b73d-1745b03f93ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.011894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-74ce5ffe-2d74-4aed-b73d-1745b03f93ce None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:47.013413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-74ce5ffe-2d74-4aed-b73d-1745b03f93ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:47.054668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 136/272] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:47.056062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8ee88-fb3a-45c7-a8a0-16abae6ab4a6 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.056353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8ee88-fb3a-45c7-a8a0-16abae6ab4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.056621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8ee88-fb3a-45c7-a8a0-16abae6ab4a6 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.058070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 137/273] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:47.059323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22383ce-c5e6-4153-966f-0172005c1289 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.059604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22383ce-c5e6-4153-966f-0172005c1289 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.059873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22383ce-c5e6-4153-966f-0172005c1289 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.061098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 138/274] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:47.061737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:47.064043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985fea9-5c36-4287-bfc4-c4291da8e4a1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.064799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985fea9-5c36-4287-bfc4-c4291da8e4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.065702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985fea9-5c36-4287-bfc4-c4291da8e4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.082563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-3985fea9-5c36-4287-bfc4-c4291da8e4a1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:47.083747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3985fea9-5c36-4287-bfc4-c4291da8e4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:47.091660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 137/275] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:47.092964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e39d2-5b85-47f8-a943-2b781486cda3 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.093416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e39d2-5b85-47f8-a943-2b781486cda3 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.093863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e39d2-5b85-47f8-a943-2b781486cda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.112176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-442e39d2-5b85-47f8-a943-2b781486cda3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:47.113389 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-442e39d2-5b85-47f8-a943-2b781486cda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:47.154505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 139/276] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:47.155557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985fea9-5c36-4287-bfc4-c4291da8e4a1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.196977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 138/277] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:47.196977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e39d2-5b85-47f8-a943-2b781486cda3 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.239239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:47.261509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985fea9-5c36-4287-bfc4-c4291da8e4a1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.286538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e39d2-5b85-47f8-a943-2b781486cda3 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.297655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:47.297853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f5a1-289b-4274-9d4c-1c9c321d2c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:47.298593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f5a1-289b-4274-9d4c-1c9c321d2c66 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.298878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f5a1-289b-4274-9d4c-1c9c321d2c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.299153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f5a1-289b-4274-9d4c-1c9c321d2c66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.300930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6f5a1-289b-4274-9d4c-1c9c321d2c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:47.302417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6f5a1-289b-4274-9d4c-1c9c321d2c66 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:47.304046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 140/278] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:47.309041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f5a1-289b-4274-9d4c-1c9c321d2c66 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.331162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c4a52-7333-4fcc-9cfe-3c6b8442663b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:47.331968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c4a52-7333-4fcc-9cfe-3c6b8442663b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.332254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c4a52-7333-4fcc-9cfe-3c6b8442663b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.332947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c4a52-7333-4fcc-9cfe-3c6b8442663b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.344019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c4a52-7333-4fcc-9cfe-3c6b8442663b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-225c4a52-7333-4fcc-9cfe-3c6b8442663b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:47.351884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:47.353025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:47.353789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:47.354379 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 139/279] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:47.355398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c4a52-7333-4fcc-9cfe-3c6b8442663b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.412340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f5a1-289b-4274-9d4c-1c9c321d2c66 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.444410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee071b2-5dc6-44f3-a606-8f74630ff0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:47.445229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee071b2-5dc6-44f3-a606-8f74630ff0ae None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.446040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee071b2-5dc6-44f3-a606-8f74630ff0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.446311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee071b2-5dc6-44f3-a606-8f74630ff0ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.456686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee071b2-5dc6-44f3-a606-8f74630ff0ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:47.458434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c4a52-7333-4fcc-9cfe-3c6b8442663b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-bee071b2-5dc6-44f3-a606-8f74630ff0ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:47.464437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:47.465519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:47.466309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 141/280] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:47.466951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee071b2-5dc6-44f3-a606-8f74630ff0ae None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.477184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:47.495660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be014abc-64ad-49d5-86ea-d1bb2b4bc442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:47.496447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be014abc-64ad-49d5-86ea-d1bb2b4bc442 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.496727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be014abc-64ad-49d5-86ea-d1bb2b4bc442 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.497030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be014abc-64ad-49d5-86ea-d1bb2b4bc442 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.507113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be014abc-64ad-49d5-86ea-d1bb2b4bc442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-be014abc-64ad-49d5-86ea-d1bb2b4bc442 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:47.514861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:47.515730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:47.516508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 140/281] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:47.517736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be014abc-64ad-49d5-86ea-d1bb2b4bc442 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.544207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:47.570048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee071b2-5dc6-44f3-a606-8f74630ff0ae None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.606419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077714d-7fcf-4480-b8e1-e61e788d9b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:47.607095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077714d-7fcf-4480-b8e1-e61e788d9b4b None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.607378 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077714d-7fcf-4480-b8e1-e61e788d9b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.607654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077714d-7fcf-4480-b8e1-e61e788d9b4b None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.608524 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077714d-7fcf-4480-b8e1-e61e788d9b4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:47.609842 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077714d-7fcf-4480-b8e1-e61e788d9b4b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:47.622167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be014abc-64ad-49d5-86ea-d1bb2b4bc442 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.630481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 142/282] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:15:47 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:47.631097 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077714d-7fcf-4480-b8e1-e61e788d9b4b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.671085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f0e07-b587-4a64-96ae-02da2525e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:47.671837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f0e07-b587-4a64-96ae-02da2525e39b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.672125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f0e07-b587-4a64-96ae-02da2525e39b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.672584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f0e07-b587-4a64-96ae-02da2525e39b None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.673416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f0e07-b587-4a64-96ae-02da2525e39b None admin] RBAC: Adding query filter params (name=member) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:47.673736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f0e07-b587-4a64-96ae-02da2525e39b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:47.675161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f0e07-b587-4a64-96ae-02da2525e39b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:47.685888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 141/283] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:47.686773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f0e07-b587-4a64-96ae-02da2525e39b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.709776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:47.754663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077714d-7fcf-4480-b8e1-e61e788d9b4b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.790407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:47.799321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b99c3a-8aba-4ecf-8e56-337fdb1b0cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:47.800268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b99c3a-8aba-4ecf-8e56-337fdb1b0cbf None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.800549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b99c3a-8aba-4ecf-8e56-337fdb1b0cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.800837 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b99c3a-8aba-4ecf-8e56-337fdb1b0cbf None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.801621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b99c3a-8aba-4ecf-8e56-337fdb1b0cbf None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:47.801925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b99c3a-8aba-4ecf-8e56-337fdb1b0cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:47.803313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b99c3a-8aba-4ecf-8e56-337fdb1b0cbf None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:47.805407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f0e07-b587-4a64-96ae-02da2525e39b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.818799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 143/284] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:47.818799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b99c3a-8aba-4ecf-8e56-337fdb1b0cbf None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.843980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1bf1a-fbe1-48bf-bb2f-521c526beb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:47.844655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1bf1a-fbe1-48bf-bb2f-521c526beb02 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.844968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1bf1a-fbe1-48bf-bb2f-521c526beb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.845246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1bf1a-fbe1-48bf-bb2f-521c526beb02 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.846355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1bf1a-fbe1-48bf-bb2f-521c526beb02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:47.846355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1bf1a-fbe1-48bf-bb2f-521c526beb02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:47.847761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1bf1a-fbe1-48bf-bb2f-521c526beb02 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:47.856576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 142/285] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:47.857291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1bf1a-fbe1-48bf-bb2f-521c526beb02 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:47.941228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b99c3a-8aba-4ecf-8e56-337fdb1b0cbf None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.967946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:47.978150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1bf1a-fbe1-48bf-bb2f-521c526beb02 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:47.985372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7929d8-0a69-4897-bd08-6debc0b457b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:47.986423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7929d8-0a69-4897-bd08-6debc0b457b1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:47.986713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7929d8-0a69-4897-bd08-6debc0b457b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:47.986988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7929d8-0a69-4897-bd08-6debc0b457b1 None admin] PATH_INFO: `/v3/users/31c9427ce622494f875b977076a7a3e2` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:47.988332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7929d8-0a69-4897-bd08-6debc0b457b1 None admin] RBAC: Authorizing `identity:get_user(user_id=31c9427ce622494f875b977076a7a3e2)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:47.989983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7929d8-0a69-4897-bd08-6debc0b457b1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:47.991692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 144/286] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/users/31c9427ce622494f875b977076a7a3e2 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:47.993278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7929d8-0a69-4897-bd08-6debc0b457b1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:48.019581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9fa27-eae9-406a-b473-09ba0352cc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:48.020704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9fa27-eae9-406a-b473-09ba0352cc6e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:48.021039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9fa27-eae9-406a-b473-09ba0352cc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:48.021315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9fa27-eae9-406a-b473-09ba0352cc6e None admin] PATH_INFO: `/v3/users/cc6734595127424a89bd096d1f89625c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:48.040502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9fa27-eae9-406a-b473-09ba0352cc6e None admin] RBAC: Authorizing `identity:get_user(user_id=cc6734595127424a89bd096d1f89625c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:48.042030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9fa27-eae9-406a-b473-09ba0352cc6e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:48.043842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 143/287] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/users/cc6734595127424a89bd096d1f89625c => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:48.048748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9fa27-eae9-406a-b473-09ba0352cc6e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:48.058024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:48.141349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7929d8-0a69-4897-bd08-6debc0b457b1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:48.165812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9fa27-eae9-406a-b473-09ba0352cc6e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:48.185035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b942a6-5686-464f-8857-ea452a9fd5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:48.185910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b942a6-5686-464f-8857-ea452a9fd5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:48.186191 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b942a6-5686-464f-8857-ea452a9fd5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:48.186492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b942a6-5686-464f-8857-ea452a9fd5d7 None admin] PATH_INFO: `/v3/users/4f1426dbd23141659191f6f1c83ed794` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:48.188117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b942a6-5686-464f-8857-ea452a9fd5d7 None admin] RBAC: Authorizing `identity:get_user(user_id=4f1426dbd23141659191f6f1c83ed794)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:48.189991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b942a6-5686-464f-8857-ea452a9fd5d7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:48.191826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 145/288] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:47 2022] GET /identity/v3/users/4f1426dbd23141659191f6f1c83ed794 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:48.192410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b942a6-5686-464f-8857-ea452a9fd5d7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:48.208531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba8632-e4dd-42fc-89f8-4c17f1dea6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:48.209232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba8632-e4dd-42fc-89f8-4c17f1dea6fd None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:48.209517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba8632-e4dd-42fc-89f8-4c17f1dea6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:48.209794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba8632-e4dd-42fc-89f8-4c17f1dea6fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:48.276986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:48.290954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba8632-e4dd-42fc-89f8-4c17f1dea6fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:48.292740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba8632-e4dd-42fc-89f8-4c17f1dea6fd None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:48.303461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:48.343423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b942a6-5686-464f-8857-ea452a9fd5d7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:48.391808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157554d4-1ba4-4387-bece-acd4ed658125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:48.392474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157554d4-1ba4-4387-bece-acd4ed658125 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:48.392753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157554d4-1ba4-4387-bece-acd4ed658125 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:48.393049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157554d4-1ba4-4387-bece-acd4ed658125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:48.394168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157554d4-1ba4-4387-bece-acd4ed658125 None admin] RBAC: Adding query filter params (role.id=8de5789832e84f9388b1295d479fca6e, user.id=cc6734595127424a89bd096d1f89625c, scope.system=all) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:48.394483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157554d4-1ba4-4387-bece-acd4ed658125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:48.396172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157554d4-1ba4-4387-bece-acd4ed658125 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:48.408025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 146/289] 10.222.0.30 () {56 vars in 1308 bytes} [Mon May 23 15:15:48 2022] GET /identity/v3/role_assignments?user.id=cc6734595127424a89bd096d1f89625c&scope.system=all&role.id=8de5789832e84f9388b1295d479fca6e => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:48.408025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157554d4-1ba4-4387-bece-acd4ed658125 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:48.439948 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 144/290] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:48.457524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba8632-e4dd-42fc-89f8-4c17f1dea6fd None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:48.539000 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157554d4-1ba4-4387-bece-acd4ed658125 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:48.568224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:48.591450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32073039-57c1-4aad-94e7-9ce497a7e4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:48.592558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32073039-57c1-4aad-94e7-9ce497a7e4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:48.592993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32073039-57c1-4aad-94e7-9ce497a7e4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:48.593398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32073039-57c1-4aad-94e7-9ce497a7e4b9 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:48.597964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32073039-57c1-4aad-94e7-9ce497a7e4b9 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:48.599678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32073039-57c1-4aad-94e7-9ce497a7e4b9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:48.602359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 147/291] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:48 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:48.612757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32073039-57c1-4aad-94e7-9ce497a7e4b9 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:48.679847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba8632-e4dd-42fc-89f8-4c17f1dea6fd None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:48.696996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:48.732640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59db291-12c3-49a5-a71a-a95199190f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:48.733850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59db291-12c3-49a5-a71a-a95199190f7b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:48.734321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59db291-12c3-49a5-a71a-a95199190f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:48.734693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59db291-12c3-49a5-a71a-a95199190f7b None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:48.736237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59db291-12c3-49a5-a71a-a95199190f7b None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:48.738328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59db291-12c3-49a5-a71a-a95199190f7b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:48.740304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 145/292] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:48 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:48.747186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59db291-12c3-49a5-a71a-a95199190f7b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:48.871405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32073039-57c1-4aad-94e7-9ce497a7e4b9 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:48.907263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59db291-12c3-49a5-a71a-a95199190f7b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:48.949450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f5277-1ca9-416f-bbb4-efe2fbb795f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:48.951491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f5277-1ca9-416f-bbb4-efe2fbb795f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:48.951785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f5277-1ca9-416f-bbb4-efe2fbb795f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:48.952062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f5277-1ca9-416f-bbb4-efe2fbb795f6 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/users/4f1426dbd23141659191f6f1c83ed794/roles/2acd8a15a63a4261b1ae12f09d28722f` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:48.956898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f5277-1ca9-416f-bbb4-efe2fbb795f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cbcf4de61f4fb59bbc787e33742b7e, user_id=4f1426dbd23141659191f6f1c83ed794, role_id=2acd8a15a63a4261b1ae12f09d28722f)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:48.958359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f5277-1ca9-416f-bbb4-efe2fbb795f6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:48.983420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 148/293] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:15:48 2022] PUT /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/users/4f1426dbd23141659191f6f1c83ed794/roles/2acd8a15a63a4261b1ae12f09d28722f => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:15:48.983839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851cf8e-1b1c-4071-9c2d-c95c4fb59d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:48.984562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851cf8e-1b1c-4071-9c2d-c95c4fb59d75 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:48.984913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851cf8e-1b1c-4071-9c2d-c95c4fb59d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:48.985197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851cf8e-1b1c-4071-9c2d-c95c4fb59d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:48.986101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851cf8e-1b1c-4071-9c2d-c95c4fb59d75 None admin] RBAC: Adding query filter params (role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e, user.id=31c9427ce622494f875b977076a7a3e2) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:48.986418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851cf8e-1b1c-4071-9c2d-c95c4fb59d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:48.988078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851cf8e-1b1c-4071-9c2d-c95c4fb59d75 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:48.995884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 146/294] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:48 2022] GET /identity/v3/role_assignments?user.id=31c9427ce622494f875b977076a7a3e2&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=8de5789832e84f9388b1295d479fca6e => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:50.102478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb40462-f517-4fe9-be20-237bd1db59a5 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.103003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb40462-f517-4fe9-be20-237bd1db59a5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.103089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb40462-f517-4fe9-be20-237bd1db59a5 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.104418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 149/295] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:50.109715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd862-d176-4491-99e2-ba97b737a130 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.110168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd862-d176-4491-99e2-ba97b737a130 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.110317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd862-d176-4491-99e2-ba97b737a130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.128019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-3a9dd862-d176-4491-99e2-ba97b737a130 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:50.129229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3a9dd862-d176-4491-99e2-ba97b737a130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:50.164411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:50.197052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb129b0-58e5-4278-a70d-daeca3e1307a None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.197451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb129b0-58e5-4278-a70d-daeca3e1307a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.197739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb129b0-58e5-4278-a70d-daeca3e1307a None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.199121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 147/296] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:50.199414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 150/297] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:50.204660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b6989-6b31-4304-9056-8b679d00b8cd None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.204968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b6989-6b31-4304-9056-8b679d00b8cd None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.205246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b6989-6b31-4304-9056-8b679d00b8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.225437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ac1b6989-6b31-4304-9056-8b679d00b8cd None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:50.227182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ac1b6989-6b31-4304-9056-8b679d00b8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:50.230958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb129b0-58e5-4278-a70d-daeca3e1307a None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:50.297859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:50.330800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 148/298] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:50.357136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b6989-6b31-4304-9056-8b679d00b8cd None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:50.359164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb129b0-58e5-4278-a70d-daeca3e1307a None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:50.419440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f922db4-30df-488e-a8af-94c792ec280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:50.420347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f922db4-30df-488e-a8af-94c792ec280f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.420692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f922db4-30df-488e-a8af-94c792ec280f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.421293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f922db4-30df-488e-a8af-94c792ec280f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.434427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f922db4-30df-488e-a8af-94c792ec280f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-7f922db4-30df-488e-a8af-94c792ec280f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:50.444324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:50.445234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:50.446753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 151/299] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:15:50.451861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f922db4-30df-488e-a8af-94c792ec280f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:50.484767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:50.490604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b6989-6b31-4304-9056-8b679d00b8cd None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:50.529247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739f773-2f90-4c00-8882-8160dfe72c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:50.530436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739f773-2f90-4c00-8882-8160dfe72c94 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.530918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739f773-2f90-4c00-8882-8160dfe72c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.531410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739f773-2f90-4c00-8882-8160dfe72c94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.532797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739f773-2f90-4c00-8882-8160dfe72c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:50.534582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739f773-2f90-4c00-8882-8160dfe72c94 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:50.540958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 149/300] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:50.543032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739f773-2f90-4c00-8882-8160dfe72c94 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:50.557005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f922db4-30df-488e-a8af-94c792ec280f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:50.570215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:50.593323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499bb9a-f43c-49ab-b5db-b9259685f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:50.594002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499bb9a-f43c-49ab-b5db-b9259685f102 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.594285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499bb9a-f43c-49ab-b5db-b9259685f102 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.594578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499bb9a-f43c-49ab-b5db-b9259685f102 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.595318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499bb9a-f43c-49ab-b5db-b9259685f102 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:50.595626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499bb9a-f43c-49ab-b5db-b9259685f102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:50.597013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499bb9a-f43c-49ab-b5db-b9259685f102 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:50.610377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 152/301] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:50.611447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499bb9a-f43c-49ab-b5db-b9259685f102 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:50.670170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739f773-2f90-4c00-8882-8160dfe72c94 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:50.716154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e99870-a848-440e-9b9d-bd2d101054cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:50.717256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e99870-a848-440e-9b9d-bd2d101054cb None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.717718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e99870-a848-440e-9b9d-bd2d101054cb None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.718179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e99870-a848-440e-9b9d-bd2d101054cb None admin] PATH_INFO: `/v3/groups` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.719154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e99870-a848-440e-9b9d-bd2d101054cb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:50.721123 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e99870-a848-440e-9b9d-bd2d101054cb None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:50.727442 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499bb9a-f43c-49ab-b5db-b9259685f102 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:50.749012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 150/302] 10.222.0.30 () {60 vars in 1134 bytes} [Mon May 23 15:15:50 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:50.767692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0100be-4092-4469-8666-276df78910cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:50.768752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0100be-4092-4469-8666-276df78910cd None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.769145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0100be-4092-4469-8666-276df78910cd None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.769493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0100be-4092-4469-8666-276df78910cd None admin] PATH_INFO: `/v3/users/1c394533b49a45fd9f6c922a3ed7c5b0` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.785960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0100be-4092-4469-8666-276df78910cd None admin] RBAC: Authorizing `identity:get_user(user_id=1c394533b49a45fd9f6c922a3ed7c5b0)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:50.787501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0100be-4092-4469-8666-276df78910cd None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:50.789438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 153/303] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:50 2022] GET /identity/v3/users/1c394533b49a45fd9f6c922a3ed7c5b0 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:50.795564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e99870-a848-440e-9b9d-bd2d101054cb None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:50.829942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:50.916092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0c5d2-3946-454d-85e2-2226568f79aa None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.916265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0c5d2-3946-454d-85e2-2226568f79aa None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.916542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0c5d2-3946-454d-85e2-2226568f79aa None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.918296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 154/304] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:50.922651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e99870-a848-440e-9b9d-bd2d101054cb None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:50.924066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6f6c2-ae9e-4c18-97bd-d1b790733081 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.924357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6f6c2-ae9e-4c18-97bd-d1b790733081 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.924632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6f6c2-ae9e-4c18-97bd-d1b790733081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.943027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-c3b6f6c2-ae9e-4c18-97bd-d1b790733081 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:50.944252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-c3b6f6c2-ae9e-4c18-97bd-d1b790733081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:50.962411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de84311-b2eb-4e49-ab4d-ceed331aed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:50.963468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de84311-b2eb-4e49-ab4d-ceed331aed0b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:50.963975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de84311-b2eb-4e49-ab4d-ceed331aed0b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:50.964455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de84311-b2eb-4e49-ab4d-ceed331aed0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:50.965627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de84311-b2eb-4e49-ab4d-ceed331aed0b None admin] RBAC: Adding query filter params (role.id=2acd8a15a63a4261b1ae12f09d28722f, user.id=1c394533b49a45fd9f6c922a3ed7c5b0, scope.system=all) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:50.966134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de84311-b2eb-4e49-ab4d-ceed331aed0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:50.967814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de84311-b2eb-4e49-ab4d-ceed331aed0b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:50.973986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:50.974254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 151/305] 10.222.0.30 () {56 vars in 1308 bytes} [Mon May 23 15:15:50 2022] GET /identity/v3/role_assignments?user.id=1c394533b49a45fd9f6c922a3ed7c5b0&scope.system=all&role.id=2acd8a15a63a4261b1ae12f09d28722f => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:51.035164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 155/306] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:51.057131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de84311-b2eb-4e49-ab4d-ceed331aed0b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:51.167703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:51.177796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de84311-b2eb-4e49-ab4d-ceed331aed0b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:51.220673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571ab38-f8b9-4666-b8e2-21707c7d648e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:51.221746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571ab38-f8b9-4666-b8e2-21707c7d648e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:51.222174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571ab38-f8b9-4666-b8e2-21707c7d648e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:51.222625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571ab38-f8b9-4666-b8e2-21707c7d648e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:51.235218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571ab38-f8b9-4666-b8e2-21707c7d648e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-7571ab38-f8b9-4666-b8e2-21707c7d648e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:51.244377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:51.245822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:51.246874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:51.248971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 152/307] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:51.255342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6f6c2-ae9e-4c18-97bd-d1b790733081 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:51.370067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b829fb-d52a-4d1f-bff5-616c71ab4249 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:51.371008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b829fb-d52a-4d1f-bff5-616c71ab4249 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:51.371533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b829fb-d52a-4d1f-bff5-616c71ab4249 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:51.371897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6f6c2-ae9e-4c18-97bd-d1b790733081 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:51.373807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 153/308] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:51.381648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdbdb0-3865-4f99-bc68-7ae90805dc31 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:51.382601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdbdb0-3865-4f99-bc68-7ae90805dc31 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:51.383155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdbdb0-3865-4f99-bc68-7ae90805dc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:51.406584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-01bdbdb0-3865-4f99-bc68-7ae90805dc31 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:51.406772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d0c4a-a25d-4143-b05e-6f633e55ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:51.407273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d0c4a-a25d-4143-b05e-6f633e55ad3c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:51.407556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d0c4a-a25d-4143-b05e-6f633e55ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:51.407830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d0c4a-a25d-4143-b05e-6f633e55ad3c None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:51.408581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d0c4a-a25d-4143-b05e-6f633e55ad3c None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:51.408901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d0c4a-a25d-4143-b05e-6f633e55ad3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:51.409409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-01bdbdb0-3865-4f99-bc68-7ae90805dc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:51.410451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d0c4a-a25d-4143-b05e-6f633e55ad3c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:51.420250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 156/309] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:51.425597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d0c4a-a25d-4143-b05e-6f633e55ad3c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:51.453279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:51.500825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 154/310] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:51.527068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdbdb0-3865-4f99-bc68-7ae90805dc31 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:51.536244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d0c4a-a25d-4143-b05e-6f633e55ad3c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:51.579597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5919a8-be3d-41ea-b089-192e24b23a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:51.581030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5919a8-be3d-41ea-b089-192e24b23a3d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:51.581585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5919a8-be3d-41ea-b089-192e24b23a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:51.582181 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5919a8-be3d-41ea-b089-192e24b23a3d None admin] PATH_INFO: `/v3/users/cc6734595127424a89bd096d1f89625c` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:51.583757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5919a8-be3d-41ea-b089-192e24b23a3d None admin] RBAC: Authorizing `identity:get_user(user_id=cc6734595127424a89bd096d1f89625c)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:51.585478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5919a8-be3d-41ea-b089-192e24b23a3d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:51.588476 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 157/311] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:51 2022] GET /identity/v3/users/cc6734595127424a89bd096d1f89625c => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:51.595319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5919a8-be3d-41ea-b089-192e24b23a3d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:51.640590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:51.666687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdbdb0-3865-4f99-bc68-7ae90805dc31 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:51.710228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd032a4-21d7-4053-81f5-24ded3742b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:51.711589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd032a4-21d7-4053-81f5-24ded3742b50 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:51.711867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd032a4-21d7-4053-81f5-24ded3742b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:51.712138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd032a4-21d7-4053-81f5-24ded3742b50 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:51.725234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:51.725951 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd032a4-21d7-4053-81f5-24ded3742b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-1bd032a4-21d7-4053-81f5-24ded3742b50 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:51.734682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:51.735540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:51.736732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 155/312] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:15:51.740265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5919a8-be3d-41ea-b089-192e24b23a3d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:51.741524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd032a4-21d7-4053-81f5-24ded3742b50 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:51.783127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32249aff-a60d-414e-a423-894aca995a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:51.783986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32249aff-a60d-414e-a423-894aca995a44 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:51.784266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32249aff-a60d-414e-a423-894aca995a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:51.784537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32249aff-a60d-414e-a423-894aca995a44 None admin] PATH_INFO: `/v3/system/users/cc6734595127424a89bd096d1f89625c/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:51.786066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32249aff-a60d-414e-a423-894aca995a44 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cc6734595127424a89bd096d1f89625c, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:51.787624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32249aff-a60d-414e-a423-894aca995a44 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:51.805476 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 158/313] 10.222.0.30 () {58 vars in 1334 bytes} [Mon May 23 15:15:51 2022] PUT /identity/v3/system/users/cc6734595127424a89bd096d1f89625c/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:15:51.877199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd032a4-21d7-4053-81f5-24ded3742b50 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:51.919414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d927c-ba7f-4759-a3d1-dfde6c64d747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:51.920534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d927c-ba7f-4759-a3d1-dfde6c64d747 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:51.921115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d927c-ba7f-4759-a3d1-dfde6c64d747 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:51.921669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d927c-ba7f-4759-a3d1-dfde6c64d747 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:51.922763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d927c-ba7f-4759-a3d1-dfde6c64d747 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:51.923300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d927c-ba7f-4759-a3d1-dfde6c64d747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:51.925095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d927c-ba7f-4759-a3d1-dfde6c64d747 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:51.935943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 156/314] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:51.941316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32249aff-a60d-414e-a423-894aca995a44 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:52.065465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32249aff-a60d-414e-a423-894aca995a44 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:52.100661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a02af-3e76-4860-92a9-a510b7240048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:52.102171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a02af-3e76-4860-92a9-a510b7240048 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:52.102441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a02af-3e76-4860-92a9-a510b7240048 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:52.102714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a02af-3e76-4860-92a9-a510b7240048 None admin] PATH_INFO: `/v3/users/4f1426dbd23141659191f6f1c83ed794` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:52.103922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a02af-3e76-4860-92a9-a510b7240048 None admin] RBAC: Authorizing `identity:get_user(user_id=4f1426dbd23141659191f6f1c83ed794)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:52.105397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a02af-3e76-4860-92a9-a510b7240048 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:52.107371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 159/315] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:51 2022] GET /identity/v3/users/4f1426dbd23141659191f6f1c83ed794 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:52.112621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d927c-ba7f-4759-a3d1-dfde6c64d747 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:52.121158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:52.222325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d927c-ba7f-4759-a3d1-dfde6c64d747 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:52.264023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68731d-34fe-4b56-8313-9e819b571da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:52.264712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68731d-34fe-4b56-8313-9e819b571da4 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:52.265054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68731d-34fe-4b56-8313-9e819b571da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:52.265335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68731d-34fe-4b56-8313-9e819b571da4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:52.343006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68731d-34fe-4b56-8313-9e819b571da4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:52.344410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68731d-34fe-4b56-8313-9e819b571da4 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:52.347113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:52.455398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 157/316] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:52.461905 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a02af-3e76-4860-92a9-a510b7240048 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:52.491952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:52.571221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a02af-3e76-4860-92a9-a510b7240048 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:52.606364 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5b3f8-5bea-4a19-8187-6dd53bb1c1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:52.607158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5b3f8-5bea-4a19-8187-6dd53bb1c1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:52.607437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5b3f8-5bea-4a19-8187-6dd53bb1c1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:52.607710 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5b3f8-5bea-4a19-8187-6dd53bb1c1e1 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:52.608922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5b3f8-5bea-4a19-8187-6dd53bb1c1e1 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:52.610368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5b3f8-5bea-4a19-8187-6dd53bb1c1e1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:52.612399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 160/317] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:52 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:52.618063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68731d-34fe-4b56-8313-9e819b571da4 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:52.724421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68731d-34fe-4b56-8313-9e819b571da4 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:52.727450 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:52.761474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c92a8-2007-42a5-9d50-30bd654b36d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:52.762603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c92a8-2007-42a5-9d50-30bd654b36d2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:52.763084 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c92a8-2007-42a5-9d50-30bd654b36d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:52.763561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c92a8-2007-42a5-9d50-30bd654b36d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:52.764615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c92a8-2007-42a5-9d50-30bd654b36d2 None admin] RBAC: Adding query filter params (role.id=2acd8a15a63a4261b1ae12f09d28722f, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e, user.id=4f1426dbd23141659191f6f1c83ed794) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:52.765174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c92a8-2007-42a5-9d50-30bd654b36d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:52.766828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c92a8-2007-42a5-9d50-30bd654b36d2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:52.780903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 158/318] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:15:52 2022] GET /identity/v3/role_assignments?user.id=4f1426dbd23141659191f6f1c83ed794&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=2acd8a15a63a4261b1ae12f09d28722f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:53.152290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca8779-d22c-463b-bf74-750a7e0175cb None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:53.152686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca8779-d22c-463b-bf74-750a7e0175cb None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:53.152862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca8779-d22c-463b-bf74-750a7e0175cb None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:53.154379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 161/319] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:53.159430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c9543-23fe-4d42-a784-c815a83c5025 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:53.159713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c9543-23fe-4d42-a784-c815a83c5025 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:53.159990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c9543-23fe-4d42-a784-c815a83c5025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:53.177175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a21c9543-23fe-4d42-a784-c815a83c5025 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:53.178331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a21c9543-23fe-4d42-a784-c815a83c5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:53.251187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 159/320] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:53.273356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca8779-d22c-463b-bf74-750a7e0175cb None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:53.372697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:53.384122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca8779-d22c-463b-bf74-750a7e0175cb None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:53.410707 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6731f7-e248-4d87-a1f5-4f6336c78a27 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:53.410937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6731f7-e248-4d87-a1f5-4f6336c78a27 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:53.411173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6731f7-e248-4d87-a1f5-4f6336c78a27 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:53.412662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 160/321] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:53.418552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559803df-93d4-4523-b32d-74efcf129637 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:53.418841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559803df-93d4-4523-b32d-74efcf129637 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:53.419119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559803df-93d4-4523-b32d-74efcf129637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:53.427106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da3844-2b86-4c24-a35b-f325ca08ec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:53.428223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da3844-2b86-4c24-a35b-f325ca08ec0a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:53.428509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da3844-2b86-4c24-a35b-f325ca08ec0a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:53.428787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da3844-2b86-4c24-a35b-f325ca08ec0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:53.430433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da3844-2b86-4c24-a35b-f325ca08ec0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:53.432605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da3844-2b86-4c24-a35b-f325ca08ec0a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:53.441101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-559803df-93d4-4523-b32d-74efcf129637 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:53.441312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 162/322] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:15:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:53.441401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-559803df-93d4-4523-b32d-74efcf129637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:53.443296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da3844-2b86-4c24-a35b-f325ca08ec0a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:53.512182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 161/323] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:53.534936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559803df-93d4-4523-b32d-74efcf129637 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:53.552739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da3844-2b86-4c24-a35b-f325ca08ec0a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:53.563600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:53.594367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb5c6b-61dc-42b7-85b6-f86568b9874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:53.595092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb5c6b-61dc-42b7-85b6-f86568b9874f None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:53.595385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb5c6b-61dc-42b7-85b6-f86568b9874f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:53.595662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb5c6b-61dc-42b7-85b6-f86568b9874f None admin] PATH_INFO: `/v3/groups` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:53.596583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb5c6b-61dc-42b7-85b6-f86568b9874f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:53.598170 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb5c6b-61dc-42b7-85b6-f86568b9874f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:53.616973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 163/324] 10.222.0.30 () {60 vars in 1134 bytes} [Mon May 23 15:15:53 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:15:53.650021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559803df-93d4-4523-b32d-74efcf129637 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:53.689372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d9020-e2b6-4015-acfb-57b4ffbaa6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:53.690094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d9020-e2b6-4015-acfb-57b4ffbaa6df None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:53.690377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d9020-e2b6-4015-acfb-57b4ffbaa6df None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:53.690636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d9020-e2b6-4015-acfb-57b4ffbaa6df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:53.702828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d9020-e2b6-4015-acfb-57b4ffbaa6df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b19d9020-e2b6-4015-acfb-57b4ffbaa6df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:53.711398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:53.712598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:53.713574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 162/325] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:15:53.722196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb5c6b-61dc-42b7-85b6-f86568b9874f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:53.755007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:53.840364 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb5c6b-61dc-42b7-85b6-f86568b9874f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:53.877579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7790dd-2ad6-4611-900d-4a85dfda9c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:53.878464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7790dd-2ad6-4611-900d-4a85dfda9c70 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:53.878846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7790dd-2ad6-4611-900d-4a85dfda9c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:53.879133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7790dd-2ad6-4611-900d-4a85dfda9c70 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:53.879923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7790dd-2ad6-4611-900d-4a85dfda9c70 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:53.880236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7790dd-2ad6-4611-900d-4a85dfda9c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:53.881689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7790dd-2ad6-4611-900d-4a85dfda9c70 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:53.891824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 164/326] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:53.898137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d9020-e2b6-4015-acfb-57b4ffbaa6df None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:54.002407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:54.016455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d9020-e2b6-4015-acfb-57b4ffbaa6df None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:54.053380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20605cf7-616d-4486-8ead-0bca89b3b23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:54.054485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20605cf7-616d-4486-8ead-0bca89b3b23c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:54.055030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20605cf7-616d-4486-8ead-0bca89b3b23c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:54.055480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20605cf7-616d-4486-8ead-0bca89b3b23c None admin] PATH_INFO: `/v3/users/1c394533b49a45fd9f6c922a3ed7c5b0` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:54.056853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20605cf7-616d-4486-8ead-0bca89b3b23c None admin] RBAC: Authorizing `identity:get_user(user_id=1c394533b49a45fd9f6c922a3ed7c5b0)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:54.058487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20605cf7-616d-4486-8ead-0bca89b3b23c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:54.060277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 163/327] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:53 2022] GET /identity/v3/users/1c394533b49a45fd9f6c922a3ed7c5b0 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:54.065895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7790dd-2ad6-4611-900d-4a85dfda9c70 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:54.083055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19440a-eef4-4b0f-abe8-1b2ff335d6b5 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:54.083620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19440a-eef4-4b0f-abe8-1b2ff335d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:54.084045 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19440a-eef4-4b0f-abe8-1b2ff335d6b5 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:54.086440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 164/328] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:54.092114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9a988-ba76-48a8-a3b4-d3e8ab882487 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:54.092611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9a988-ba76-48a8-a3b4-d3e8ab882487 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:54.093125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9a988-ba76-48a8-a3b4-d3e8ab882487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:54.111810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-9eb9a988-ba76-48a8-a3b4-d3e8ab882487 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:54.113372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-9eb9a988-ba76-48a8-a3b4-d3e8ab882487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:54.172573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:54.195875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7790dd-2ad6-4611-900d-4a85dfda9c70 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:54.202254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 165/329] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:54.227236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9a988-ba76-48a8-a3b4-d3e8ab882487 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:54.253925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9172426-73bf-4669-9281-9706ab43b9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:54.254712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9172426-73bf-4669-9281-9706ab43b9a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:54.254991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9172426-73bf-4669-9281-9706ab43b9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:54.255355 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9172426-73bf-4669-9281-9706ab43b9a2 None admin] PATH_INFO: `/v3/system/users/1c394533b49a45fd9f6c922a3ed7c5b0/roles/2acd8a15a63a4261b1ae12f09d28722f` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:54.257025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9172426-73bf-4669-9281-9706ab43b9a2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1c394533b49a45fd9f6c922a3ed7c5b0, role_id=2acd8a15a63a4261b1ae12f09d28722f)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:54.259143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9172426-73bf-4669-9281-9706ab43b9a2 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:54.267441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 165/330] 10.222.0.30 () {58 vars in 1334 bytes} [Mon May 23 15:15:54 2022] PUT /identity/v3/system/users/1c394533b49a45fd9f6c922a3ed7c5b0/roles/2acd8a15a63a4261b1ae12f09d28722f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:15:54.346885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:54.365917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9a988-ba76-48a8-a3b4-d3e8ab882487 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:54.408200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a848d6-dbdc-45b6-ae35-865192b85c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:54.409034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a848d6-dbdc-45b6-ae35-865192b85c35 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:54.409317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a848d6-dbdc-45b6-ae35-865192b85c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:54.409597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a848d6-dbdc-45b6-ae35-865192b85c35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:54.422352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a848d6-dbdc-45b6-ae35-865192b85c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-e3a848d6-dbdc-45b6-ae35-865192b85c35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:54.431391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:54.432330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:54.433124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 166/331] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:15:54.439690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9172426-73bf-4669-9281-9706ab43b9a2 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:54.564378 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9172426-73bf-4669-9281-9706ab43b9a2 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:54.613243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e195-647b-44e6-9507-14d0cf5e61b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:54.614227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e195-647b-44e6-9507-14d0cf5e61b8 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:54.614748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e195-647b-44e6-9507-14d0cf5e61b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:54.615246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e195-647b-44e6-9507-14d0cf5e61b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:54.616218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5e195-647b-44e6-9507-14d0cf5e61b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:54.616724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5e195-647b-44e6-9507-14d0cf5e61b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:54.619165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5e195-647b-44e6-9507-14d0cf5e61b8 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:54.630441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 166/332] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:54.635697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a848d6-dbdc-45b6-ae35-865192b85c35 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:54.753806 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a848d6-dbdc-45b6-ae35-865192b85c35 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:54.803837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd869c-cd2c-4c12-9871-7550f286f349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:54.804807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd869c-cd2c-4c12-9871-7550f286f349 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:54.805113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd869c-cd2c-4c12-9871-7550f286f349 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:54.805388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd869c-cd2c-4c12-9871-7550f286f349 None admin] PATH_INFO: `/v3/users/cc6734595127424a89bd096d1f89625c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:54.809593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd869c-cd2c-4c12-9871-7550f286f349 None admin] RBAC: Authorizing `identity:get_user(user_id=cc6734595127424a89bd096d1f89625c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:54.811076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd869c-cd2c-4c12-9871-7550f286f349 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:54.812739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 167/333] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:54 2022] GET /identity/v3/users/cc6734595127424a89bd096d1f89625c => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:54.818592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e195-647b-44e6-9507-14d0cf5e61b8 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:54.843537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:54.976851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e195-647b-44e6-9507-14d0cf5e61b8 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:55.012605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ec147-baa8-4ee9-a7cd-0a442c89a91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:55.013302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ec147-baa8-4ee9-a7cd-0a442c89a91d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:55.013587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ec147-baa8-4ee9-a7cd-0a442c89a91d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:55.013867 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ec147-baa8-4ee9-a7cd-0a442c89a91d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:55.014690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ec147-baa8-4ee9-a7cd-0a442c89a91d None admin] RBAC: Adding query filter params (role.id=8de5789832e84f9388b1295d479fca6e, user.id=cc6734595127424a89bd096d1f89625c, scope.system=all) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:55.014997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ec147-baa8-4ee9-a7cd-0a442c89a91d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:55.016435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ec147-baa8-4ee9-a7cd-0a442c89a91d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:55.032435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 167/334] 10.222.0.30 () {56 vars in 1308 bytes} [Mon May 23 15:15:54 2022] GET /identity/v3/role_assignments?user.id=cc6734595127424a89bd096d1f89625c&scope.system=all&role.id=8de5789832e84f9388b1295d479fca6e => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:56.550989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efbdd8-bbd6-496a-bb1c-c5750a11e2e4 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:56.551545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efbdd8-bbd6-496a-bb1c-c5750a11e2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:56.551545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efbdd8-bbd6-496a-bb1c-c5750a11e2e4 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:56.553205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 168/335] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:56.559605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e0bb6-82e7-41b9-a235-323c380d2b96 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:56.560077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e0bb6-82e7-41b9-a235-323c380d2b96 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:56.560160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e0bb6-82e7-41b9-a235-323c380d2b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:56.583389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-bc7e0bb6-82e7-41b9-a235-323c380d2b96 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:56.584586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-bc7e0bb6-82e7-41b9-a235-323c380d2b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:56.670390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:56.678968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 168/336] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:56.723701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efbdd8-bbd6-496a-bb1c-c5750a11e2e4 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:56.758085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:56.882636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efbdd8-bbd6-496a-bb1c-c5750a11e2e4 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:56.920749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561b61f-2cac-4369-8fb6-01ea3ce24fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:56.921933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561b61f-2cac-4369-8fb6-01ea3ce24fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:56.922427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561b61f-2cac-4369-8fb6-01ea3ce24fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:56.922912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561b61f-2cac-4369-8fb6-01ea3ce24fc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:56.934015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561b61f-2cac-4369-8fb6-01ea3ce24fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-a561b61f-2cac-4369-8fb6-01ea3ce24fc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:56.942082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:56.942980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:56.943802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:56.947167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 169/337] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:56.957990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e0bb6-82e7-41b9-a235-323c380d2b96 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:57.107612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e0bb6-82e7-41b9-a235-323c380d2b96 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:57.150446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179ac8-2aa4-4f81-b9d4-a2955bed62db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:57.151491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179ac8-2aa4-4f81-b9d4-a2955bed62db None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:57.151974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179ac8-2aa4-4f81-b9d4-a2955bed62db None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:57.152447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179ac8-2aa4-4f81-b9d4-a2955bed62db None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:57.153417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179ac8-2aa4-4f81-b9d4-a2955bed62db None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:57.153941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179ac8-2aa4-4f81-b9d4-a2955bed62db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:57.155531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179ac8-2aa4-4f81-b9d4-a2955bed62db None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:57.169438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 169/338] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:57.176059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561b61f-2cac-4369-8fb6-01ea3ce24fc3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:57.282227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:57.318188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561b61f-2cac-4369-8fb6-01ea3ce24fc3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:57.366889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9beb63-d841-4798-a711-e7c03a108c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:57.368330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9beb63-d841-4798-a711-e7c03a108c05 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:57.368610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9beb63-d841-4798-a711-e7c03a108c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:57.368895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9beb63-d841-4798-a711-e7c03a108c05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:57.457485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9beb63-d841-4798-a711-e7c03a108c05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:57.458875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9beb63-d841-4798-a711-e7c03a108c05 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:57.474838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e788f4-5205-49ba-8e03-3373504afe83 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:57.475308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e788f4-5205-49ba-8e03-3373504afe83 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:57.475595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e788f4-5205-49ba-8e03-3373504afe83 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:57.477629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 170/339] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:57.486542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11d7d2-7149-4738-be86-780a3dbf6f9e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:57.486836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11d7d2-7149-4738-be86-780a3dbf6f9e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:57.487092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11d7d2-7149-4738-be86-780a3dbf6f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:57.533394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4f11d7d2-7149-4738-be86-780a3dbf6f9e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:57.534590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4f11d7d2-7149-4738-be86-780a3dbf6f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:57.622064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:57.660850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:57.661374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 170/340] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:57.668666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9beb63-d841-4798-a711-e7c03a108c05 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:57.687425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 171/341] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:57.688570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d3ad8-b7f1-491f-98be-708573087710 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:57.688857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d3ad8-b7f1-491f-98be-708573087710 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:57.689214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d3ad8-b7f1-491f-98be-708573087710 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:57.690747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 172/342] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:57.702637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4517be4-eba9-4581-bdfc-2fbd4d243345 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:57.703250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4517be4-eba9-4581-bdfc-2fbd4d243345 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:57.703744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4517be4-eba9-4581-bdfc-2fbd4d243345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:57.729139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a4517be4-eba9-4581-bdfc-2fbd4d243345 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:57.730655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a4517be4-eba9-4581-bdfc-2fbd4d243345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:57.811030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 173/343] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:15:57.812230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4517be4-eba9-4581-bdfc-2fbd4d243345 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:57.823006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9beb63-d841-4798-a711-e7c03a108c05 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:57.881537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cf666-4551-4a11-9aae-db3510549ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:57.882472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cf666-4551-4a11-9aae-db3510549ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:57.882930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cf666-4551-4a11-9aae-db3510549ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:57.883260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cf666-4551-4a11-9aae-db3510549ce3 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:57.885191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cf666-4551-4a11-9aae-db3510549ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:57.886678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cf666-4551-4a11-9aae-db3510549ce3 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:57.894194 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 171/344] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:57 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:15:57.895121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cf666-4551-4a11-9aae-db3510549ce3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:57.927221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4517be4-eba9-4581-bdfc-2fbd4d243345 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:57.948038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:57.967859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621faa3-9370-43e9-876f-0cbf065f1543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:57.969018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621faa3-9370-43e9-876f-0cbf065f1543 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:57.969504 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621faa3-9370-43e9-876f-0cbf065f1543 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:57.970086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621faa3-9370-43e9-876f-0cbf065f1543 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:57.983939 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621faa3-9370-43e9-876f-0cbf065f1543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-6621faa3-9370-43e9-876f-0cbf065f1543 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:57.994739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:57.995935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:15:57.996828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:57.997497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:57.999870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 174/345] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:58.001195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621faa3-9370-43e9-876f-0cbf065f1543 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:58.102303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cf666-4551-4a11-9aae-db3510549ce3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:58.134562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621faa3-9370-43e9-876f-0cbf065f1543 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:58.163871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedfd5f-7897-4725-a281-7002f03478d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:58.165305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedfd5f-7897-4725-a281-7002f03478d8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.165612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedfd5f-7897-4725-a281-7002f03478d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.165887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedfd5f-7897-4725-a281-7002f03478d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.178037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc1ced-f357-4c54-ad97-ff2ecd526ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:58.179197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc1ced-f357-4c54-ad97-ff2ecd526ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.179700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc1ced-f357-4c54-ad97-ff2ecd526ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.180166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc1ced-f357-4c54-ad97-ff2ecd526ba6 None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.187395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aedfd5f-7897-4725-a281-7002f03478d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-8aedfd5f-7897-4725-a281-7002f03478d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:58.196161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:58.197113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:58.198004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:58.201606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc1ced-f357-4c54-ad97-ff2ecd526ba6 None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:58.203258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc1ced-f357-4c54-ad97-ff2ecd526ba6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:58.205192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 175/346] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:15:58 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:58.206135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc1ced-f357-4c54-ad97-ff2ecd526ba6 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:58.210376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 172/347] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:15:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:58.213667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedfd5f-7897-4725-a281-7002f03478d8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:58.272551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:58.354146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedfd5f-7897-4725-a281-7002f03478d8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:58.356093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:58.391845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc1ced-f357-4c54-ad97-ff2ecd526ba6 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:58.397954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56ae8d-d943-4334-bc84-9ec450eaf934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:58.399356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56ae8d-d943-4334-bc84-9ec450eaf934 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.399785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56ae8d-d943-4334-bc84-9ec450eaf934 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.400203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56ae8d-d943-4334-bc84-9ec450eaf934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.401307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56ae8d-d943-4334-bc84-9ec450eaf934 None admin] RBAC: Adding query filter params (group.id=b6c303a21aec4dc1b49b7cc15d2eb063, role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=84b308b5ae9a4e0482f37715f327cd20) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:58.401630 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56ae8d-d943-4334-bc84-9ec450eaf934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:58.403113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56ae8d-d943-4334-bc84-9ec450eaf934 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:58.410095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 173/348] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:15:58 2022] GET /identity/v3/role_assignments?group.id=b6c303a21aec4dc1b49b7cc15d2eb063&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=8de5789832e84f9388b1295d479fca6e => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:58.411097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56ae8d-d943-4334-bc84-9ec450eaf934 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:58.430015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be788f93-441a-4522-a21e-010b76167458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:58.430870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788f93-441a-4522-a21e-010b76167458 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.431169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788f93-441a-4522-a21e-010b76167458 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.431563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788f93-441a-4522-a21e-010b76167458 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.432391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be788f93-441a-4522-a21e-010b76167458 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:58.432858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be788f93-441a-4522-a21e-010b76167458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:58.434642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be788f93-441a-4522-a21e-010b76167458 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:58.445723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 176/349] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:58.447941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93af8d1-c46f-44cc-b960-e15467402edd None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.448840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93af8d1-c46f-44cc-b960-e15467402edd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.449682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93af8d1-c46f-44cc-b960-e15467402edd None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.452006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 177/350] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:58.457588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46123c5f-4e70-491b-b13a-a40358d41f19 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.458456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46123c5f-4e70-491b-b13a-a40358d41f19 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.458751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46123c5f-4e70-491b-b13a-a40358d41f19 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.460267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 178/351] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:58.461905 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46123c5f-4e70-491b-b13a-a40358d41f19 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:58.560007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56ae8d-d943-4334-bc84-9ec450eaf934 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:58.625342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2d0fe-fe84-4fea-87dc-d70870ac7b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:58.626117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2d0fe-fe84-4fea-87dc-d70870ac7b7b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.626326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2d0fe-fe84-4fea-87dc-d70870ac7b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.626611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2d0fe-fe84-4fea-87dc-d70870ac7b7b None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.627387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2d0fe-fe84-4fea-87dc-d70870ac7b7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:58.627696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2d0fe-fe84-4fea-87dc-d70870ac7b7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:58.629494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2d0fe-fe84-4fea-87dc-d70870ac7b7b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:58.640455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 174/352] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:15:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:58.641900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46123c5f-4e70-491b-b13a-a40358d41f19 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:58.642099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690d6dd-3fc0-4fa5-bee0-0622ab289df0 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.642390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690d6dd-3fc0-4fa5-bee0-0622ab289df0 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.642673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690d6dd-3fc0-4fa5-bee0-0622ab289df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.646282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:58.668940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-5690d6dd-3fc0-4fa5-bee0-0622ab289df0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:58.670485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5690d6dd-3fc0-4fa5-bee0-0622ab289df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:58.683103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9327e-7844-4aa5-89ce-6de3c16b7173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:58.684661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9327e-7844-4aa5-89ce-6de3c16b7173 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.685177 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9327e-7844-4aa5-89ce-6de3c16b7173 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.685689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9327e-7844-4aa5-89ce-6de3c16b7173 None admin] PATH_INFO: `/v3/users/1c394533b49a45fd9f6c922a3ed7c5b0` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.687209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9327e-7844-4aa5-89ce-6de3c16b7173 None admin] RBAC: Authorizing `identity:get_user(user_id=1c394533b49a45fd9f6c922a3ed7c5b0)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:58.688838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9327e-7844-4aa5-89ce-6de3c16b7173 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:58.690690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 179/353] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:15:58 2022] GET /identity/v3/users/1c394533b49a45fd9f6c922a3ed7c5b0 => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:58.692080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14b286-1f2c-42ea-b4a7-399e87ac5be2 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.692590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14b286-1f2c-42ea-b4a7-399e87ac5be2 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.693090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14b286-1f2c-42ea-b4a7-399e87ac5be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.701229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:58.711684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-2d14b286-1f2c-42ea-b4a7-399e87ac5be2 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:58.713241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2d14b286-1f2c-42ea-b4a7-399e87ac5be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:58.765420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 175/354] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:58.766964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690d6dd-3fc0-4fa5-bee0-0622ab289df0 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:58.816845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 180/355] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:58.817983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14b286-1f2c-42ea-b4a7-399e87ac5be2 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:58.890872 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690d6dd-3fc0-4fa5-bee0-0622ab289df0 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:58.936490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5de62-c56f-4715-a664-d265880e6385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:58.937197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5de62-c56f-4715-a664-d265880e6385 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.937732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5de62-c56f-4715-a664-d265880e6385 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.938218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5de62-c56f-4715-a664-d265880e6385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.943970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14b286-1f2c-42ea-b4a7-399e87ac5be2 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:58.946326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:58.975993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:58.988400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb8b5a-6590-481a-96ac-3075e0ee4827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:58.989372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8b5a-6590-481a-96ac-3075e0ee4827 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:58.989794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8b5a-6590-481a-96ac-3075e0ee4827 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:58.990232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8b5a-6590-481a-96ac-3075e0ee4827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:58.991227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb8b5a-6590-481a-96ac-3075e0ee4827 None admin] RBAC: Adding query filter params (role.id=2acd8a15a63a4261b1ae12f09d28722f, user.id=1c394533b49a45fd9f6c922a3ed7c5b0, scope.system=all) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:58.992662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb8b5a-6590-481a-96ac-3075e0ee4827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:58.994712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb8b5a-6590-481a-96ac-3075e0ee4827 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:59.015390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 181/356] 10.222.0.30 () {56 vars in 1308 bytes} [Mon May 23 15:15:58 2022] GET /identity/v3/role_assignments?user.id=1c394533b49a45fd9f6c922a3ed7c5b0&scope.system=all&role.id=2acd8a15a63a4261b1ae12f09d28722f => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:59.017158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43304b24-473c-4374-bbd9-12454397beb1 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.017744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43304b24-473c-4374-bbd9-12454397beb1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.018200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43304b24-473c-4374-bbd9-12454397beb1 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:59.019812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 182/357] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:15:59.025038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43304b24-473c-4374-bbd9-12454397beb1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:59.029803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5de62-c56f-4715-a664-d265880e6385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:59.031332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5de62-c56f-4715-a664-d265880e6385 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:59.157951 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 176/358] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:15:59.158932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5de62-c56f-4715-a664-d265880e6385 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:59.164622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43304b24-473c-4374-bbd9-12454397beb1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:59.212037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470a739-60c3-4b6b-ba30-20aa5d7f8e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:59.213282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a739-60c3-4b6b-ba30-20aa5d7f8e01 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.213567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a739-60c3-4b6b-ba30-20aa5d7f8e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.213840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a739-60c3-4b6b-ba30-20aa5d7f8e01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:59.228171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470a739-60c3-4b6b-ba30-20aa5d7f8e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:59.233141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-3470a739-60c3-4b6b-ba30-20aa5d7f8e01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:59.239537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:59.240450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:15:59.242102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 183/359] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:15:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:59.243443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e8e91-9257-4c22-a2ab-9e14a21bce8b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.243837 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e8e91-9257-4c22-a2ab-9e14a21bce8b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.244086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e8e91-9257-4c22-a2ab-9e14a21bce8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:59.266783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a97e8e91-9257-4c22-a2ab-9e14a21bce8b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:15:59.267992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a97e8e91-9257-4c22-a2ab-9e14a21bce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:15:59.286664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:59.296085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5de62-c56f-4715-a664-d265880e6385 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:59.345466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350dbe2-7479-468a-853c-9c58101efda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:59.346276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350dbe2-7479-468a-853c-9c58101efda5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.346551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350dbe2-7479-468a-853c-9c58101efda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.346818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350dbe2-7479-468a-853c-9c58101efda5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:59.361792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350dbe2-7479-468a-853c-9c58101efda5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:59.364362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 184/360] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:15:59.365287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e8e91-9257-4c22-a2ab-9e14a21bce8b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-2350dbe2-7479-468a-853c-9c58101efda5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:59.372153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:59.373072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:59.373834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 177/361] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:15:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:59.376358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350dbe2-7479-468a-853c-9c58101efda5 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:59.498366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e8e91-9257-4c22-a2ab-9e14a21bce8b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:59.510414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350dbe2-7479-468a-853c-9c58101efda5 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:59.531946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:59.545056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb3737-faa3-48ae-a054-d8d074ac57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:59.545931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb3737-faa3-48ae-a054-d8d074ac57fe None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.546210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb3737-faa3-48ae-a054-d8d074ac57fe None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.546479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb3737-faa3-48ae-a054-d8d074ac57fe None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:59.547666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb3737-faa3-48ae-a054-d8d074ac57fe None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:59.549168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb3737-faa3-48ae-a054-d8d074ac57fe None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:59.551908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 185/362] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:15:59 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:59.551908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb3737-faa3-48ae-a054-d8d074ac57fe None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:59.559341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab6a2b-a726-4629-a8bc-567a9c52c38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:59.560017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab6a2b-a726-4629-a8bc-567a9c52c38c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.560301 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab6a2b-a726-4629-a8bc-567a9c52c38c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.560576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab6a2b-a726-4629-a8bc-567a9c52c38c None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:59.561369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab6a2b-a726-4629-a8bc-567a9c52c38c None admin] RBAC: Adding query filter params (name=member) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:59.561678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab6a2b-a726-4629-a8bc-567a9c52c38c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:59.563075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab6a2b-a726-4629-a8bc-567a9c52c38c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:59.574504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 178/363] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:15:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:59.575145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab6a2b-a726-4629-a8bc-567a9c52c38c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:59.579292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:59.684363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb3737-faa3-48ae-a054-d8d074ac57fe None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:59.718308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab6a2b-a726-4629-a8bc-567a9c52c38c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:59.723990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aef7f-45cd-48df-a8ac-c7a315327549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:59.725312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aef7f-45cd-48df-a8ac-c7a315327549 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.725312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aef7f-45cd-48df-a8ac-c7a315327549 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.725675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aef7f-45cd-48df-a8ac-c7a315327549 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:59.726875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aef7f-45cd-48df-a8ac-c7a315327549 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:15:59.727381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aef7f-45cd-48df-a8ac-c7a315327549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:59.728813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aef7f-45cd-48df-a8ac-c7a315327549 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:59.740261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 186/364] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:15:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:59.741498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aef7f-45cd-48df-a8ac-c7a315327549 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:59.773219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e6a49-8c7e-4403-a0e9-fc0f640e190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:59.773219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e6a49-8c7e-4403-a0e9-fc0f640e190b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.773219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e6a49-8c7e-4403-a0e9-fc0f640e190b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.773219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e6a49-8c7e-4403-a0e9-fc0f640e190b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:59.786716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e6a49-8c7e-4403-a0e9-fc0f640e190b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-ab0e6a49-8c7e-4403-a0e9-fc0f640e190b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:15:59.798240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:15:59.799187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:15:59.799972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 179/365] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:15:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:15:59.800634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e6a49-8c7e-4403-a0e9-fc0f640e190b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:59.854914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:59.866905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:15:59.884851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aef7f-45cd-48df-a8ac-c7a315327549 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:59.932689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaaecc-b0ec-4961-8468-3ffe3cb16b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:59.933584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaaecc-b0ec-4961-8468-3ffe3cb16b0d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.933861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaaecc-b0ec-4961-8468-3ffe3cb16b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.934130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaaecc-b0ec-4961-8468-3ffe3cb16b0d None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:15:59.935285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aaaecc-b0ec-4961-8468-3ffe3cb16b0d None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:15:59.936698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aaaecc-b0ec-4961-8468-3ffe3cb16b0d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:15:59.938518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 187/366] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:15:59 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:15:59.939116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaaecc-b0ec-4961-8468-3ffe3cb16b0d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:15:59.941200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e6a49-8c7e-4403-a0e9-fc0f640e190b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:15:59.986566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83d925-d7db-42cf-aedd-78e9f39d43ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:15:59.987213 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83d925-d7db-42cf-aedd-78e9f39d43ea None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:15:59.987490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83d925-d7db-42cf-aedd-78e9f39d43ea None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:15:59.987759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83d925-d7db-42cf-aedd-78e9f39d43ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:00.070461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaaecc-b0ec-4961-8468-3ffe3cb16b0d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:00.074987 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83d925-d7db-42cf-aedd-78e9f39d43ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:00.076501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83d925-d7db-42cf-aedd-78e9f39d43ea None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:00.115608 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5971d-72ac-4034-963e-31e71b98ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:00.116414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5971d-72ac-4034-963e-31e71b98ea56 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:00.116702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5971d-72ac-4034-963e-31e71b98ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:00.117712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5971d-72ac-4034-963e-31e71b98ea56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:00.127800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:00.139690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:00.205331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5971d-72ac-4034-963e-31e71b98ea56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:00.205537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 180/367] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:00.206233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83d925-d7db-42cf-aedd-78e9f39d43ea None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:00.207035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5971d-72ac-4034-963e-31e71b98ea56 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:00.331467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 188/368] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:15:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:00.332595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5971d-72ac-4034-963e-31e71b98ea56 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:00.341601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83d925-d7db-42cf-aedd-78e9f39d43ea None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:00.394294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2ac2f-fb19-4149-89ad-cef781724275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:00.394626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:00.395097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2ac2f-fb19-4149-89ad-cef781724275 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:00.395760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2ac2f-fb19-4149-89ad-cef781724275 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:00.396156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2ac2f-fb19-4149-89ad-cef781724275 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:00.396919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2ac2f-fb19-4149-89ad-cef781724275 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:00.397263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2ac2f-fb19-4149-89ad-cef781724275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:00.398886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2ac2f-fb19-4149-89ad-cef781724275 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:00.408408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 181/369] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:00.409419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2ac2f-fb19-4149-89ad-cef781724275 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:00.413970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:00.465332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5971d-72ac-4034-963e-31e71b98ea56 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:00.515867 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af419660-3c98-4fe7-a16e-8c8021899414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:00.516530 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419660-3c98-4fe7-a16e-8c8021899414 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:00.516807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419660-3c98-4fe7-a16e-8c8021899414 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:00.517110 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419660-3c98-4fe7-a16e-8c8021899414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:00.517956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af419660-3c98-4fe7-a16e-8c8021899414 None admin] RBAC: Adding query filter params (group.id=b6c303a21aec4dc1b49b7cc15d2eb063, role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=84b308b5ae9a4e0482f37715f327cd20) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:00.518265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af419660-3c98-4fe7-a16e-8c8021899414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:00.519714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af419660-3c98-4fe7-a16e-8c8021899414 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:00.528328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 189/370] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:16:00 2022] GET /identity/v3/role_assignments?group.id=b6c303a21aec4dc1b49b7cc15d2eb063&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:00.529013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af419660-3c98-4fe7-a16e-8c8021899414 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:00.567906 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2ac2f-fb19-4149-89ad-cef781724275 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:00.614272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af142c6-944b-4bae-857e-22357d300923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:00.615143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af142c6-944b-4bae-857e-22357d300923 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:00.615428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af142c6-944b-4bae-857e-22357d300923 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:00.615707 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af142c6-944b-4bae-857e-22357d300923 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:00.617098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af142c6-944b-4bae-857e-22357d300923 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:00.618619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af142c6-944b-4bae-857e-22357d300923 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:00.620848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 182/371] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:00 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:00.623106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af142c6-944b-4bae-857e-22357d300923 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:00.649559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af419660-3c98-4fe7-a16e-8c8021899414 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:00.667497 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:00.694707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60189c41-6d06-4e66-abff-8a99220055da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:00.696397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189c41-6d06-4e66-abff-8a99220055da None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:00.696684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189c41-6d06-4e66-abff-8a99220055da None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:00.697000 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189c41-6d06-4e66-abff-8a99220055da None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:00.698411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60189c41-6d06-4e66-abff-8a99220055da None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:00.699997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60189c41-6d06-4e66-abff-8a99220055da None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:00.702287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 190/372] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:00 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:00.703135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60189c41-6d06-4e66-abff-8a99220055da None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:00.726798 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:00.777338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af142c6-944b-4bae-857e-22357d300923 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:00.823889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244da20-420f-469b-98e0-794a6aae70d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:00.824555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244da20-420f-469b-98e0-794a6aae70d0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:00.824885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244da20-420f-469b-98e0-794a6aae70d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:00.825147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244da20-420f-469b-98e0-794a6aae70d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:00.842696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60189c41-6d06-4e66-abff-8a99220055da None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:00.879399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bcb2-9197-4574-a689-0d489abe4e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:00.880186 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bcb2-9197-4574-a689-0d489abe4e6f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:00.880466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bcb2-9197-4574-a689-0d489abe4e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:00.880740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bcb2-9197-4574-a689-0d489abe4e6f None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:00.881991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371bcb2-9197-4574-a689-0d489abe4e6f None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:00.884009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371bcb2-9197-4574-a689-0d489abe4e6f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:00.886129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 191/373] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:00 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:00.887093 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bcb2-9197-4574-a689-0d489abe4e6f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:00.914707 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244da20-420f-469b-98e0-794a6aae70d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:00.916232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244da20-420f-469b-98e0-794a6aae70d0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:00.946920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:01.002491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:01.012842 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bcb2-9197-4574-a689-0d489abe4e6f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:01.047976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 183/374] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:01.048832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244da20-420f-469b-98e0-794a6aae70d0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:01.058921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7694f-4c2b-403e-b6ef-190f0095ef32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:01.059909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7694f-4c2b-403e-b6ef-190f0095ef32 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.060197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7694f-4c2b-403e-b6ef-190f0095ef32 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.060474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7694f-4c2b-403e-b6ef-190f0095ef32 None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.062820 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7694f-4c2b-403e-b6ef-190f0095ef32 None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:01.064270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7694f-4c2b-403e-b6ef-190f0095ef32 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:01.065966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 192/375] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:00 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:01.067231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067b9d0-87d6-4121-a554-8b68ce887a94 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.067522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067b9d0-87d6-4121-a554-8b68ce887a94 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.067801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067b9d0-87d6-4121-a554-8b68ce887a94 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.069997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 193/376] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:01.069997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067b9d0-87d6-4121-a554-8b68ce887a94 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:01.180621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244da20-420f-469b-98e0-794a6aae70d0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:01.209690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067b9d0-87d6-4121-a554-8b68ce887a94 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:01.223489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58cfc8-9e7e-4476-b84b-498db32c12bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:01.224453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58cfc8-9e7e-4476-b84b-498db32c12bd None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.224960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58cfc8-9e7e-4476-b84b-498db32c12bd None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.225437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58cfc8-9e7e-4476-b84b-498db32c12bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.226602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58cfc8-9e7e-4476-b84b-498db32c12bd None admin] RBAC: Adding query filter params (group.id=b6c303a21aec4dc1b49b7cc15d2eb063, role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:01.227112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58cfc8-9e7e-4476-b84b-498db32c12bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:01.228766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58cfc8-9e7e-4476-b84b-498db32c12bd None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:01.229216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:01.234911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 184/377] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:16:01 2022] GET /identity/v3/role_assignments?group.id=b6c303a21aec4dc1b49b7cc15d2eb063&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=8de5789832e84f9388b1295d479fca6e => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:01.236101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58cfc8-9e7e-4476-b84b-498db32c12bd None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:01.276568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19f716-f155-4c96-bcc5-09906c7572d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:01.277812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f716-f155-4c96-bcc5-09906c7572d0 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.278232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f716-f155-4c96-bcc5-09906c7572d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.278665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f716-f155-4c96-bcc5-09906c7572d0 None admin] PATH_INFO: `/v3/projects/c586cdfef8dc4cea9cf389817d345b08` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.283010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19f716-f155-4c96-bcc5-09906c7572d0 None admin] RBAC: Authorizing `identity:get_project(project_id=c586cdfef8dc4cea9cf389817d345b08)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:01.283763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:01.287304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19f716-f155-4c96-bcc5-09906c7572d0 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:01.290587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 194/378] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:01 2022] GET /identity/v3/projects/c586cdfef8dc4cea9cf389817d345b08 => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:01.292267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278103cf-30be-4be9-affb-81d2f863b464 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.292778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278103cf-30be-4be9-affb-81d2f863b464 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.293249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278103cf-30be-4be9-affb-81d2f863b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.327310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-278103cf-30be-4be9-affb-81d2f863b464 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:01.328600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-278103cf-30be-4be9-affb-81d2f863b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:01.360851 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58cfc8-9e7e-4476-b84b-498db32c12bd None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:01.403380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58400fd-8f63-4ba4-b772-1c39b8cf4b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:01.404692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58400fd-8f63-4ba4-b772-1c39b8cf4b9a None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.405076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58400fd-8f63-4ba4-b772-1c39b8cf4b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.405632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58400fd-8f63-4ba4-b772-1c39b8cf4b9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.406501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58400fd-8f63-4ba4-b772-1c39b8cf4b9a None admin] RBAC: Adding query filter params (group.id=b6c303a21aec4dc1b49b7cc15d2eb063, role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:01.406978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58400fd-8f63-4ba4-b772-1c39b8cf4b9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:01.409847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58400fd-8f63-4ba4-b772-1c39b8cf4b9a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:01.419065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 185/379] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:16:01 2022] GET /identity/v3/role_assignments?group.id=b6c303a21aec4dc1b49b7cc15d2eb063&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:01.419787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 195/380] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:01.419868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58400fd-8f63-4ba4-b772-1c39b8cf4b9a None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:01.453629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278103cf-30be-4be9-affb-81d2f863b464 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:01.577411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:01.584252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58400fd-8f63-4ba4-b772-1c39b8cf4b9a None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:01.622705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278103cf-30be-4be9-affb-81d2f863b464 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:01.654165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:01.661592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e9c28-6407-46c5-bf46-fb69780c9708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:01.662456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e9c28-6407-46c5-bf46-fb69780c9708 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.662733 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e9c28-6407-46c5-bf46-fb69780c9708 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.663000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e9c28-6407-46c5-bf46-fb69780c9708 None admin] PATH_INFO: `/v3/groups/f3eb814f1aad49369d340c6264d6c39a` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.677882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e9c28-6407-46c5-bf46-fb69780c9708 None admin] RBAC: Authorizing `identity:get_group(group_id=f3eb814f1aad49369d340c6264d6c39a)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:01.679352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e9c28-6407-46c5-bf46-fb69780c9708 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:01.681696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 186/381] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:01 2022] GET /identity/v3/groups/f3eb814f1aad49369d340c6264d6c39a => generated 229 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:01.687855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e9c28-6407-46c5-bf46-fb69780c9708 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:01.694037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653af58-a303-42d6-93e9-0a46fa9ddcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:01.694832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653af58-a303-42d6-93e9-0a46fa9ddcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.695402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653af58-a303-42d6-93e9-0a46fa9ddcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.695402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653af58-a303-42d6-93e9-0a46fa9ddcb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.716737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653af58-a303-42d6-93e9-0a46fa9ddcb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-4653af58-a303-42d6-93e9-0a46fa9ddcb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:01.724386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:01.725290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:16:01.726054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:01.728127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 196/382] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:01.739785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653af58-a303-42d6-93e9-0a46fa9ddcb8 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:01.884282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653af58-a303-42d6-93e9-0a46fa9ddcb8 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:01.922527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e9c28-6407-46c5-bf46-fb69780c9708 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:01.923851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2ccb8-2470-4193-b4bd-a505584a648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:01.924781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2ccb8-2470-4193-b4bd-a505584a648c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.925088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2ccb8-2470-4193-b4bd-a505584a648c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.925358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2ccb8-2470-4193-b4bd-a505584a648c None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.926111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2ccb8-2470-4193-b4bd-a505584a648c None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:01.926412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2ccb8-2470-4193-b4bd-a505584a648c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:01.927796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2ccb8-2470-4193-b4bd-a505584a648c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:01.938858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 197/383] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:16:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:01.966314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2ccb8-2470-4193-b4bd-a505584a648c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:01.989679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffeaa5-fe39-478e-b2cb-ec0431e9df6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:01.990333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffeaa5-fe39-478e-b2cb-ec0431e9df6b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:01.990717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffeaa5-fe39-478e-b2cb-ec0431e9df6b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:01.991003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffeaa5-fe39-478e-b2cb-ec0431e9df6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:01.992429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffeaa5-fe39-478e-b2cb-ec0431e9df6b None admin] RBAC: Adding query filter params (group.id=f3eb814f1aad49369d340c6264d6c39a, role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=c586cdfef8dc4cea9cf389817d345b08) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:01.992754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffeaa5-fe39-478e-b2cb-ec0431e9df6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:01.994376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffeaa5-fe39-478e-b2cb-ec0431e9df6b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:02.005293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:02.012361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 187/384] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:16:01 2022] GET /identity/v3/role_assignments?group.id=f3eb814f1aad49369d340c6264d6c39a&scope.project.id=c586cdfef8dc4cea9cf389817d345b08&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 260 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:02.076990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2ccb8-2470-4193-b4bd-a505584a648c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:02.111922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54682989-0ac8-4367-9a7f-cec730c0d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:02.112786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682989-0ac8-4367-9a7f-cec730c0d507 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:02.113093 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682989-0ac8-4367-9a7f-cec730c0d507 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:02.113367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682989-0ac8-4367-9a7f-cec730c0d507 None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:02.114553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54682989-0ac8-4367-9a7f-cec730c0d507 None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:02.115983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54682989-0ac8-4367-9a7f-cec730c0d507 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:02.118474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 198/385] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:01 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:02.124274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffeaa5-fe39-478e-b2cb-ec0431e9df6b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:02.155648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:02.232454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffeaa5-fe39-478e-b2cb-ec0431e9df6b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:02.270436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab006b-1550-4b77-a9af-e1872978cb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:02.271232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab006b-1550-4b77-a9af-e1872978cb5d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:02.271662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab006b-1550-4b77-a9af-e1872978cb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:02.272421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab006b-1550-4b77-a9af-e1872978cb5d None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:02.284893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab006b-1550-4b77-a9af-e1872978cb5d None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:02.284893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab006b-1550-4b77-a9af-e1872978cb5d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:02.284893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 188/386] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:02 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:02.287406 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54682989-0ac8-4367-9a7f-cec730c0d507 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:02.497727 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54682989-0ac8-4367-9a7f-cec730c0d507 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:02.501222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:02.548647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8a522-6599-45fb-ab6b-b7cc88c35d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:02.549522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8a522-6599-45fb-ab6b-b7cc88c35d39 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:02.549806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8a522-6599-45fb-ab6b-b7cc88c35d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:02.550081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8a522-6599-45fb-ab6b-b7cc88c35d39 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20/groups/b6c303a21aec4dc1b49b7cc15d2eb063/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:02.558608 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8a522-6599-45fb-ab6b-b7cc88c35d39 None admin] RBAC: Authorizing `identity:create_grant(project_id=84b308b5ae9a4e0482f37715f327cd20, group_id=b6c303a21aec4dc1b49b7cc15d2eb063, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:02.560090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8a522-6599-45fb-ab6b-b7cc88c35d39 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:02.574954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 199/387] 10.222.0.30 () {58 vars in 1442 bytes} [Mon May 23 15:16:02 2022] PUT /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20/groups/b6c303a21aec4dc1b49b7cc15d2eb063/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:03.187104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a3134-ab96-40ee-81b4-0ffce87d7c75 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:03.187635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a3134-ab96-40ee-81b4-0ffce87d7c75 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:03.187635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a3134-ab96-40ee-81b4-0ffce87d7c75 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:03.189263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 189/388] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:03.197536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e788f-1a53-4925-aeff-90511d05a8b9 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:03.197536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e788f-1a53-4925-aeff-90511d05a8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:03.197536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e788f-1a53-4925-aeff-90511d05a8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:03.219458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-fd4e788f-1a53-4925-aeff-90511d05a8b9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:03.221043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fd4e788f-1a53-4925-aeff-90511d05a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:03.309516 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 200/389] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:03.342273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a3134-ab96-40ee-81b4-0ffce87d7c75 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:03.488187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:03.498480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a3134-ab96-40ee-81b4-0ffce87d7c75 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:03.552314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5206d4-18e4-4932-89e2-4dd14c973e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:03.553156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5206d4-18e4-4932-89e2-4dd14c973e27 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:03.553436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5206d4-18e4-4932-89e2-4dd14c973e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:03.553705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5206d4-18e4-4932-89e2-4dd14c973e27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:03.565058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5206d4-18e4-4932-89e2-4dd14c973e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-fb5206d4-18e4-4932-89e2-4dd14c973e27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:03.573754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:03.574621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:03.575613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 190/390] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:16:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:03.580639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e788f-1a53-4925-aeff-90511d05a8b9 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:03.747528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:03.759412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e788f-1a53-4925-aeff-90511d05a8b9 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:03.809660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c9ff5-53c4-40e9-8073-625eed32e63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:03.810744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9ff5-53c4-40e9-8073-625eed32e63c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:03.811230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9ff5-53c4-40e9-8073-625eed32e63c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:03.811724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9ff5-53c4-40e9-8073-625eed32e63c None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:03.812687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c9ff5-53c4-40e9-8073-625eed32e63c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:03.813268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c9ff5-53c4-40e9-8073-625eed32e63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:03.814843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c9ff5-53c4-40e9-8073-625eed32e63c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:03.830129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 201/391] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:03.835791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5206d4-18e4-4932-89e2-4dd14c973e27 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:03.845395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710359aa-bc35-4a64-92c8-a4c17081c53a None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:03.845688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710359aa-bc35-4a64-92c8-a4c17081c53a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:03.845972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710359aa-bc35-4a64-92c8-a4c17081c53a None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:03.848290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 202/392] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:03.853508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13833dcd-5c70-4b53-85d0-ab067b19b754 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:03.853877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13833dcd-5c70-4b53-85d0-ab067b19b754 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:03.854351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13833dcd-5c70-4b53-85d0-ab067b19b754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:03.885065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-13833dcd-5c70-4b53-85d0-ab067b19b754 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:03.886266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-13833dcd-5c70-4b53-85d0-ab067b19b754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:03.980662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 203/393] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:04.003604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13833dcd-5c70-4b53-85d0-ab067b19b754 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:04.039919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5206d4-18e4-4932-89e2-4dd14c973e27 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:04.099221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879da60e-04ca-4c1d-8224-ba93d53a2acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:04.100075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879da60e-04ca-4c1d-8224-ba93d53a2acf None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:04.100357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879da60e-04ca-4c1d-8224-ba93d53a2acf None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:04.100632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879da60e-04ca-4c1d-8224-ba93d53a2acf None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:04.102390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879da60e-04ca-4c1d-8224-ba93d53a2acf None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:04.103848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879da60e-04ca-4c1d-8224-ba93d53a2acf None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:04.105821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 191/394] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:03 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:04.111387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879da60e-04ca-4c1d-8224-ba93d53a2acf None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:04.129295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:04.160158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13833dcd-5c70-4b53-85d0-ab067b19b754 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:04.162232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:04.226187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4f2de-a03c-4eb8-999b-ccabea480e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:04.227432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4f2de-a03c-4eb8-999b-ccabea480e0a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:04.227735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4f2de-a03c-4eb8-999b-ccabea480e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:04.228031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4f2de-a03c-4eb8-999b-ccabea480e0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:04.256846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4f2de-a03c-4eb8-999b-ccabea480e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-f1a4f2de-a03c-4eb8-999b-ccabea480e0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:04.272135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:04.273079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:16:04.273901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:04.281461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 204/395] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:04.287299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4f2de-a03c-4eb8-999b-ccabea480e0a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:04.357260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879da60e-04ca-4c1d-8224-ba93d53a2acf None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:04.416546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4f2de-a03c-4eb8-999b-ccabea480e0a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:04.460244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193652f-7572-4e52-9b11-c57447b035d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:04.461548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193652f-7572-4e52-9b11-c57447b035d5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:04.461840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193652f-7572-4e52-9b11-c57447b035d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:04.462118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193652f-7572-4e52-9b11-c57447b035d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:04.462874 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193652f-7572-4e52-9b11-c57447b035d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:04.463184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193652f-7572-4e52-9b11-c57447b035d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:04.464546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193652f-7572-4e52-9b11-c57447b035d5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:04.467786 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d93b4-645b-4bfe-8efa-d613f09d7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:04.468655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d93b4-645b-4bfe-8efa-d613f09d7c8f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:04.469154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d93b4-645b-4bfe-8efa-d613f09d7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:04.469433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d93b4-645b-4bfe-8efa-d613f09d7c8f None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:04.470639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d93b4-645b-4bfe-8efa-d613f09d7c8f None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:04.472085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d93b4-645b-4bfe-8efa-d613f09d7c8f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:04.473804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 192/396] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:04 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:04.479815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d93b4-645b-4bfe-8efa-d613f09d7c8f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:04.489248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 205/397] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:16:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:04.495923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193652f-7572-4e52-9b11-c57447b035d5 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:04.517840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:04.633185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:04.646555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193652f-7572-4e52-9b11-c57447b035d5 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:04.646757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d93b4-645b-4bfe-8efa-d613f09d7c8f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:04.684205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c61b28-f186-4cb7-be3a-2f282b6112c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:04.685479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c61b28-f186-4cb7-be3a-2f282b6112c7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:04.685959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c61b28-f186-4cb7-be3a-2f282b6112c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:04.686384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c61b28-f186-4cb7-be3a-2f282b6112c7 None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:04.687890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c61b28-f186-4cb7-be3a-2f282b6112c7 None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:04.689607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c61b28-f186-4cb7-be3a-2f282b6112c7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:04.690943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d5f76-84ba-40f1-ae55-477eeeeeb719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:04.691921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d5f76-84ba-40f1-ae55-477eeeeeb719 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:04.692096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 206/398] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:04 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:04.692240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d5f76-84ba-40f1-ae55-477eeeeeb719 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:04.692599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d5f76-84ba-40f1-ae55-477eeeeeb719 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20/groups/b6c303a21aec4dc1b49b7cc15d2eb063/roles/e2a3f2f08d18410a954a4d6b8314562d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:04.693850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73903dd-2804-4ece-b742-915c42a90c9b None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:04.694295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73903dd-2804-4ece-b742-915c42a90c9b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:04.694726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73903dd-2804-4ece-b742-915c42a90c9b None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:04.694836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d5f76-84ba-40f1-ae55-477eeeeeb719 None admin] RBAC: Authorizing `identity:create_grant(project_id=84b308b5ae9a4e0482f37715f327cd20, group_id=b6c303a21aec4dc1b49b7cc15d2eb063, role_id=e2a3f2f08d18410a954a4d6b8314562d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:04.700711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d5f76-84ba-40f1-ae55-477eeeeeb719 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:04.701191 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 207/399] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:04.702269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73903dd-2804-4ece-b742-915c42a90c9b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:04.714071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 193/400] 10.222.0.30 () {58 vars in 1442 bytes} [Mon May 23 15:16:04 2022] PUT /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20/groups/b6c303a21aec4dc1b49b7cc15d2eb063/roles/e2a3f2f08d18410a954a4d6b8314562d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:04.715790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0582a6-da04-4a3a-8ad8-a37e0938aa16 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:04.716343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0582a6-da04-4a3a-8ad8-a37e0938aa16 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:04.716895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0582a6-da04-4a3a-8ad8-a37e0938aa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:04.738685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-9a0582a6-da04-4a3a-8ad8-a37e0938aa16 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:04.740657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-9a0582a6-da04-4a3a-8ad8-a37e0938aa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:04.835503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 194/401] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:04.864214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:04.880181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0582a6-da04-4a3a-8ad8-a37e0938aa16 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:04.894517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73903dd-2804-4ece-b742-915c42a90c9b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:04.938955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8e02c-d40e-41aa-9c98-d1585adacddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:04.939679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8e02c-d40e-41aa-9c98-d1585adacddd None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:04.940210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8e02c-d40e-41aa-9c98-d1585adacddd None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:04.940495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8e02c-d40e-41aa-9c98-d1585adacddd None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:04.943152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8e02c-d40e-41aa-9c98-d1585adacddd None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:04.944611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8e02c-d40e-41aa-9c98-d1585adacddd None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:04.948961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 208/402] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:04 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:04.954690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8e02c-d40e-41aa-9c98-d1585adacddd None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:05.039801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:05.067148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0582a6-da04-4a3a-8ad8-a37e0938aa16 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:05.097854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8e02c-d40e-41aa-9c98-d1585adacddd None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:05.117934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31281e16-155e-4ed0-963a-ffd8827627c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:05.119443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31281e16-155e-4ed0-963a-ffd8827627c9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.119910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31281e16-155e-4ed0-963a-ffd8827627c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.120606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31281e16-155e-4ed0-963a-ffd8827627c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.136078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31281e16-155e-4ed0-963a-ffd8827627c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:05.136723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c46e0-88b7-4a2c-b21e-33dfb363d6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:05.137714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c46e0-88b7-4a2c-b21e-33dfb363d6d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.138062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c46e0-88b7-4a2c-b21e-33dfb363d6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.138409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c46e0-88b7-4a2c-b21e-33dfb363d6d9 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/groups/b6c303a21aec4dc1b49b7cc15d2eb063/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.140637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c46e0-88b7-4a2c-b21e-33dfb363d6d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cbcf4de61f4fb59bbc787e33742b7e, group_id=b6c303a21aec4dc1b49b7cc15d2eb063, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:05.142175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c46e0-88b7-4a2c-b21e-33dfb363d6d9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-31281e16-155e-4ed0-963a-ffd8827627c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:05.147222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:05.148748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:05.150731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 195/403] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:16:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:05.151759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1f978-bbe2-446e-9197-c18e18a8f796 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.152010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1f978-bbe2-446e-9197-c18e18a8f796 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.152336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1f978-bbe2-446e-9197-c18e18a8f796 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.154674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 196/404] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:05.155416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1f978-bbe2-446e-9197-c18e18a8f796 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:05.162346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 209/405] 10.222.0.30 () {58 vars in 1442 bytes} [Mon May 23 15:16:04 2022] PUT /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/groups/b6c303a21aec4dc1b49b7cc15d2eb063/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:05.169552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14bf34-c757-45dd-a2f1-110379aa56fd None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.170198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14bf34-c757-45dd-a2f1-110379aa56fd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.170703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14bf34-c757-45dd-a2f1-110379aa56fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.186291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:05.207387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-8f14bf34-c757-45dd-a2f1-110379aa56fd None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:05.209100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8f14bf34-c757-45dd-a2f1-110379aa56fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:05.293131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1f978-bbe2-446e-9197-c18e18a8f796 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:05.335018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec71863-bf7e-4d58-9089-d7831df3ce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:05.336092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec71863-bf7e-4d58-9089-d7831df3ce58 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.336666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec71863-bf7e-4d58-9089-d7831df3ce58 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.337131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec71863-bf7e-4d58-9089-d7831df3ce58 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.338072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec71863-bf7e-4d58-9089-d7831df3ce58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:05.338581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec71863-bf7e-4d58-9089-d7831df3ce58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:05.340479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec71863-bf7e-4d58-9089-d7831df3ce58 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:05.352163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 197/406] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:05.357617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec71863-bf7e-4d58-9089-d7831df3ce58 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:05.361175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:05.391295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 210/407] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:05.395765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4010-b4bf-4653-bd2b-00e3ac81bbde None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.396293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4010-b4bf-4653-bd2b-00e3ac81bbde None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.396751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4010-b4bf-4653-bd2b-00e3ac81bbde None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.398541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 211/408] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:05.417160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7705-6185-40d3-8502-59056bc714c8 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.417913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7705-6185-40d3-8502-59056bc714c8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.418398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7705-6185-40d3-8502-59056bc714c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.449300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d82e7705-6185-40d3-8502-59056bc714c8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:05.451691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d82e7705-6185-40d3-8502-59056bc714c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:05.485505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec71863-bf7e-4d58-9089-d7831df3ce58 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:05.525127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018c9c1-324b-4e1f-a019-36c67efd21b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:05.526006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018c9c1-324b-4e1f-a019-36c67efd21b9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.526279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018c9c1-324b-4e1f-a019-36c67efd21b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.526569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018c9c1-324b-4e1f-a019-36c67efd21b9 None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.527939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018c9c1-324b-4e1f-a019-36c67efd21b9 None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:05.529455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018c9c1-324b-4e1f-a019-36c67efd21b9 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:05.531457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 198/409] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:05 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:05.532290 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018c9c1-324b-4e1f-a019-36c67efd21b9 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:05.549251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 212/410] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:05.550451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7705-6185-40d3-8502-59056bc714c8 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:05.636367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:05.646840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:05.678640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7705-6185-40d3-8502-59056bc714c8 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:05.682294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018c9c1-324b-4e1f-a019-36c67efd21b9 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:05.715859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c197d0-9eaa-4dc8-85da-b991213f7c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:05.716720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c197d0-9eaa-4dc8-85da-b991213f7c50 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.717036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c197d0-9eaa-4dc8-85da-b991213f7c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.717307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c197d0-9eaa-4dc8-85da-b991213f7c50 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.718634 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c197d0-9eaa-4dc8-85da-b991213f7c50 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:05.720153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c197d0-9eaa-4dc8-85da-b991213f7c50 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:05.720810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2795ffd-06cd-4ee9-affe-19a919467d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:05.721775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2795ffd-06cd-4ee9-affe-19a919467d7c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.722062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 213/411] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:05 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:05.722366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2795ffd-06cd-4ee9-affe-19a919467d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.722779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2795ffd-06cd-4ee9-affe-19a919467d7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.723085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c197d0-9eaa-4dc8-85da-b991213f7c50 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:05.734894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2795ffd-06cd-4ee9-affe-19a919467d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b2795ffd-06cd-4ee9-affe-19a919467d7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:05.743979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:05.744960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:05.746030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:05.748157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 199/412] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:05.753620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2795ffd-06cd-4ee9-affe-19a919467d7c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:05.830455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c197d0-9eaa-4dc8-85da-b991213f7c50 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:05.861317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:05.866576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c1f0d-734d-4277-860f-08e7905bae5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:05.868088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2795ffd-06cd-4ee9-affe-19a919467d7c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:05.868237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c1f0d-734d-4277-860f-08e7905bae5f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.868555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c1f0d-734d-4277-860f-08e7905bae5f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.868957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c1f0d-734d-4277-860f-08e7905bae5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.886425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c1f0d-734d-4277-860f-08e7905bae5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-861c1f0d-734d-4277-860f-08e7905bae5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:05.895139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:05.896714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:16:05.899253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:05.900162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 214/413] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:05.904757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c1f0d-734d-4277-860f-08e7905bae5f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:05.912576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:05.920754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f1bb7-ebeb-46d3-b943-ada8e1c36692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:05.922088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f1bb7-ebeb-46d3-b943-ada8e1c36692 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:05.922600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f1bb7-ebeb-46d3-b943-ada8e1c36692 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:05.923046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f1bb7-ebeb-46d3-b943-ada8e1c36692 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/groups/b6c303a21aec4dc1b49b7cc15d2eb063/roles/e2a3f2f08d18410a954a4d6b8314562d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:05.926161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f1bb7-ebeb-46d3-b943-ada8e1c36692 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cbcf4de61f4fb59bbc787e33742b7e, group_id=b6c303a21aec4dc1b49b7cc15d2eb063, role_id=e2a3f2f08d18410a954a4d6b8314562d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:05.928390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f1bb7-ebeb-46d3-b943-ada8e1c36692 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:05.946337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 200/414] 10.222.0.30 () {58 vars in 1442 bytes} [Mon May 23 15:16:05 2022] PUT /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e/groups/b6c303a21aec4dc1b49b7cc15d2eb063/roles/e2a3f2f08d18410a954a4d6b8314562d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:05.947478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f1bb7-ebeb-46d3-b943-ada8e1c36692 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:06.034919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c1f0d-734d-4277-860f-08e7905bae5f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:06.116486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92f928-8bfa-4c3e-8144-5b55ee754e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:06.117697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92f928-8bfa-4c3e-8144-5b55ee754e19 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:06.118194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92f928-8bfa-4c3e-8144-5b55ee754e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:06.119877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92f928-8bfa-4c3e-8144-5b55ee754e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:06.121324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92f928-8bfa-4c3e-8144-5b55ee754e19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:06.123010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92f928-8bfa-4c3e-8144-5b55ee754e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:06.125799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92f928-8bfa-4c3e-8144-5b55ee754e19 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:06.145435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 215/415] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:06.145940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f1bb7-ebeb-46d3-b943-ada8e1c36692 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:06.153136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92f928-8bfa-4c3e-8144-5b55ee754e19 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:06.214637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29986b-7c47-4fc5-aba7-f3c44d52b392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:06.215436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29986b-7c47-4fc5-aba7-f3c44d52b392 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:06.216364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29986b-7c47-4fc5-aba7-f3c44d52b392 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:06.216364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29986b-7c47-4fc5-aba7-f3c44d52b392 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:06.217061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29986b-7c47-4fc5-aba7-f3c44d52b392 None admin] RBAC: Adding query filter params (name=member) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:06.217435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29986b-7c47-4fc5-aba7-f3c44d52b392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:06.219163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29986b-7c47-4fc5-aba7-f3c44d52b392 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:06.230582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 201/416] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:16:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:06.238841 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:06.239804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29986b-7c47-4fc5-aba7-f3c44d52b392 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:06.301906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92f928-8bfa-4c3e-8144-5b55ee754e19 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:06.310143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:06.347552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81591da5-fd23-4d2a-b93b-5ed322bfdf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:06.348888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81591da5-fd23-4d2a-b93b-5ed322bfdf5f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:06.349178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81591da5-fd23-4d2a-b93b-5ed322bfdf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:06.349454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81591da5-fd23-4d2a-b93b-5ed322bfdf5f None admin] PATH_INFO: `/v3/groups/f3eb814f1aad49369d340c6264d6c39a` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:06.350886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81591da5-fd23-4d2a-b93b-5ed322bfdf5f None admin] RBAC: Authorizing `identity:get_group(group_id=f3eb814f1aad49369d340c6264d6c39a)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:06.353274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81591da5-fd23-4d2a-b93b-5ed322bfdf5f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:06.354908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 216/417] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:06 2022] GET /identity/v3/groups/f3eb814f1aad49369d340c6264d6c39a => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:06.360134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81591da5-fd23-4d2a-b93b-5ed322bfdf5f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:06.371429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29986b-7c47-4fc5-aba7-f3c44d52b392 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:06.409833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a2c4b-7099-4bfd-a012-0b76dc316f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:06.410955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a2c4b-7099-4bfd-a012-0b76dc316f7c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:06.411555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a2c4b-7099-4bfd-a012-0b76dc316f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:06.412026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a2c4b-7099-4bfd-a012-0b76dc316f7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:06.462998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81591da5-fd23-4d2a-b93b-5ed322bfdf5f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:06.486723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:06.491131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a2c4b-7099-4bfd-a012-0b76dc316f7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:06.494509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a2c4b-7099-4bfd-a012-0b76dc316f7c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:06.517607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72d197-3e66-410e-9955-221af3466a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:06.518477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72d197-3e66-410e-9955-221af3466a8a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:06.518756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72d197-3e66-410e-9955-221af3466a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:06.519025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72d197-3e66-410e-9955-221af3466a8a None admin] PATH_INFO: `/v3/projects/c586cdfef8dc4cea9cf389817d345b08` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:06.520586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72d197-3e66-410e-9955-221af3466a8a None admin] RBAC: Authorizing `identity:get_project(project_id=c586cdfef8dc4cea9cf389817d345b08)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:06.521688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72d197-3e66-410e-9955-221af3466a8a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:06.524201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 217/418] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:06 2022] GET /identity/v3/projects/c586cdfef8dc4cea9cf389817d345b08 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:06.529603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72d197-3e66-410e-9955-221af3466a8a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:06.556039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:06.622487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 202/419] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:06.628253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a2c4b-7099-4bfd-a012-0b76dc316f7c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:06.732720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a2c4b-7099-4bfd-a012-0b76dc316f7c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:06.768520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8da42-40d0-4349-9d26-da8d4a048ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:06.769894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8da42-40d0-4349-9d26-da8d4a048ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:06.770393 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8da42-40d0-4349-9d26-da8d4a048ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:06.770861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8da42-40d0-4349-9d26-da8d4a048ab1 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:06.772246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8da42-40d0-4349-9d26-da8d4a048ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:06.774156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8da42-40d0-4349-9d26-da8d4a048ab1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:06.776092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 203/420] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:06 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:06.781578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8da42-40d0-4349-9d26-da8d4a048ab1 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:06.791242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72d197-3e66-410e-9955-221af3466a8a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:06.813719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:06.889133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8da42-40d0-4349-9d26-da8d4a048ab1 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:06.909581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07044f5b-5d0d-4dd2-b79c-9351affba5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:06.910484 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07044f5b-5d0d-4dd2-b79c-9351affba5fb None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:06.910769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07044f5b-5d0d-4dd2-b79c-9351affba5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:06.911050 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07044f5b-5d0d-4dd2-b79c-9351affba5fb None admin] PATH_INFO: `/v3/projects/c586cdfef8dc4cea9cf389817d345b08/groups/f3eb814f1aad49369d340c6264d6c39a/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:06.921652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07044f5b-5d0d-4dd2-b79c-9351affba5fb None admin] RBAC: Authorizing `identity:create_grant(project_id=c586cdfef8dc4cea9cf389817d345b08, group_id=f3eb814f1aad49369d340c6264d6c39a, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:06.923098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07044f5b-5d0d-4dd2-b79c-9351affba5fb None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:06.932657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495047db-6af6-4839-a212-1136f57bd650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:06.933927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495047db-6af6-4839-a212-1136f57bd650 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:06.934408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495047db-6af6-4839-a212-1136f57bd650 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:06.934874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495047db-6af6-4839-a212-1136f57bd650 None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:06.936492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495047db-6af6-4839-a212-1136f57bd650 None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:06.938239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495047db-6af6-4839-a212-1136f57bd650 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:06.940349 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 204/421] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:06 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:06.946835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495047db-6af6-4839-a212-1136f57bd650 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:06.967376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 218/422] 10.222.0.30 () {58 vars in 1442 bytes} [Mon May 23 15:16:06 2022] PUT /identity/v3/projects/c586cdfef8dc4cea9cf389817d345b08/groups/f3eb814f1aad49369d340c6264d6c39a/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:16:07.045142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:07.059059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495047db-6af6-4839-a212-1136f57bd650 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:07.096497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36decf69-41a2-4e72-8986-e196f6ac0749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:07.097206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36decf69-41a2-4e72-8986-e196f6ac0749 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:07.097486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36decf69-41a2-4e72-8986-e196f6ac0749 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:07.097761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36decf69-41a2-4e72-8986-e196f6ac0749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:07.098597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36decf69-41a2-4e72-8986-e196f6ac0749 None admin] RBAC: Adding query filter params (group.id=b6c303a21aec4dc1b49b7cc15d2eb063, role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=84b308b5ae9a4e0482f37715f327cd20) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:07.098911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36decf69-41a2-4e72-8986-e196f6ac0749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:07.100355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36decf69-41a2-4e72-8986-e196f6ac0749 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:07.107928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 205/423] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:16:06 2022] GET /identity/v3/role_assignments?group.id=b6c303a21aec4dc1b49b7cc15d2eb063&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=8de5789832e84f9388b1295d479fca6e => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:07.368800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52cc55-caab-4c15-a1bd-124d76d8af59 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:07.369218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52cc55-caab-4c15-a1bd-124d76d8af59 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:07.369393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52cc55-caab-4c15-a1bd-124d76d8af59 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:07.371334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 219/424] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:07.376754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f1f05-a64a-4704-9f19-64e4a3dc9ea8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:07.377243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f1f05-a64a-4704-9f19-64e4a3dc9ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:07.377341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f1f05-a64a-4704-9f19-64e4a3dc9ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:07.394914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-5d8f1f05-a64a-4704-9f19-64e4a3dc9ea8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:07.396122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5d8f1f05-a64a-4704-9f19-64e4a3dc9ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:07.479656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 206/425] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:07.507359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52cc55-caab-4c15-a1bd-124d76d8af59 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:07.567954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:07.643747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52cc55-caab-4c15-a1bd-124d76d8af59 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:07.680168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a7133-dc13-47fc-aeda-0e54c0786f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:07.681294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7133-dc13-47fc-aeda-0e54c0786f85 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:07.681851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7133-dc13-47fc-aeda-0e54c0786f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:07.682270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7133-dc13-47fc-aeda-0e54c0786f85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:07.694382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a7133-dc13-47fc-aeda-0e54c0786f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-284a7133-dc13-47fc-aeda-0e54c0786f85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:07.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:07.705067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:07.706047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:07.708310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 220/426] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:16:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:07.715525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f1f05-a64a-4704-9f19-64e4a3dc9ea8 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:07.814855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:07.837062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f1f05-a64a-4704-9f19-64e4a3dc9ea8 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:07.875378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8843c1-595a-4f59-a957-c0d4558a39db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:07.875968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8843c1-595a-4f59-a957-c0d4558a39db None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:07.876250 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8843c1-595a-4f59-a957-c0d4558a39db None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:07.876523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8843c1-595a-4f59-a957-c0d4558a39db None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:07.877367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8843c1-595a-4f59-a957-c0d4558a39db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:07.877753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8843c1-595a-4f59-a957-c0d4558a39db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:07.879133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8843c1-595a-4f59-a957-c0d4558a39db None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:07.887965 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 207/427] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:07.893582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a7133-dc13-47fc-aeda-0e54c0786f85 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:07.914591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d5396-88f7-4717-974e-a612210bddc8 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:07.914591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d5396-88f7-4717-974e-a612210bddc8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:07.914591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d5396-88f7-4717-974e-a612210bddc8 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:07.916265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 208/428] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:07.921253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fd582-64f3-4784-b615-63a33434a072 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:07.921555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fd582-64f3-4784-b615-63a33434a072 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:07.921896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fd582-64f3-4784-b615-63a33434a072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:07.939816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-225fd582-64f3-4784-b615-63a33434a072 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:07.941146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-225fd582-64f3-4784-b615-63a33434a072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:08.029525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a7133-dc13-47fc-aeda-0e54c0786f85 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:08.034724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 209/429] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:08.046350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:08.071233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fd582-64f3-4784-b615-63a33434a072 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:08.075369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5644bc4-4a7e-491d-8962-9e3a578161f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:08.076169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5644bc4-4a7e-491d-8962-9e3a578161f6 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:08.076549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5644bc4-4a7e-491d-8962-9e3a578161f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:08.076948 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5644bc4-4a7e-491d-8962-9e3a578161f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:08.171251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5644bc4-4a7e-491d-8962-9e3a578161f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:08.172313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:08.173683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5644bc4-4a7e-491d-8962-9e3a578161f6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:08.200753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fd582-64f3-4784-b615-63a33434a072 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:08.241659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0b356-742e-4536-98c3-5485ffea64d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:08.242485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0b356-742e-4536-98c3-5485ffea64d6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:08.242765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0b356-742e-4536-98c3-5485ffea64d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:08.243041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0b356-742e-4536-98c3-5485ffea64d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:08.253836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0b356-742e-4536-98c3-5485ffea64d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-89a0b356-742e-4536-98c3-5485ffea64d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:08.262166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:08.263300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:08.264400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 210/430] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:08.274842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0b356-742e-4536-98c3-5485ffea64d6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:08.298799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 221/431] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:08.300458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5644bc4-4a7e-491d-8962-9e3a578161f6 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:08.311938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:08.387921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0b356-742e-4536-98c3-5485ffea64d6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:08.412065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5644bc4-4a7e-491d-8962-9e3a578161f6 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:08.425609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9583243-c015-48f2-90b8-fc5e68187b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:08.426240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9583243-c015-48f2-90b8-fc5e68187b8b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:08.426487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9583243-c015-48f2-90b8-fc5e68187b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:08.426769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9583243-c015-48f2-90b8-fc5e68187b8b None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:08.427514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9583243-c015-48f2-90b8-fc5e68187b8b None admin] RBAC: Adding query filter params (name=member) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:08.427818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9583243-c015-48f2-90b8-fc5e68187b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:08.429230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9583243-c015-48f2-90b8-fc5e68187b8b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:08.438269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 211/432] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:16:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:08.441195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:08.449855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9583243-c015-48f2-90b8-fc5e68187b8b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:08.458052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32328052-2cbd-4cdf-9606-1865fc3edef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:08.458918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32328052-2cbd-4cdf-9606-1865fc3edef4 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:08.459196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32328052-2cbd-4cdf-9606-1865fc3edef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:08.459472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32328052-2cbd-4cdf-9606-1865fc3edef4 None admin] PATH_INFO: `/v3/projects/84b308b5ae9a4e0482f37715f327cd20` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:08.460700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32328052-2cbd-4cdf-9606-1865fc3edef4 None admin] RBAC: Authorizing `identity:get_project(project_id=84b308b5ae9a4e0482f37715f327cd20)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:08.462748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32328052-2cbd-4cdf-9606-1865fc3edef4 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:08.469079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 222/433] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:08 2022] GET /identity/v3/projects/84b308b5ae9a4e0482f37715f327cd20 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:08.470349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32328052-2cbd-4cdf-9606-1865fc3edef4 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:08.560696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:08.582851 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9583243-c015-48f2-90b8-fc5e68187b8b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:08.588248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32328052-2cbd-4cdf-9606-1865fc3edef4 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:08.622188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d29bf-5fa1-499e-8e2e-bbe2b40d5bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:08.622846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d29bf-5fa1-499e-8e2e-bbe2b40d5bae None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:08.623127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d29bf-5fa1-499e-8e2e-bbe2b40d5bae None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:08.623402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d29bf-5fa1-499e-8e2e-bbe2b40d5bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:08.628799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25df13a-f13e-4ae1-96f0-a02d369230f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:08.629846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25df13a-f13e-4ae1-96f0-a02d369230f4 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:08.630293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25df13a-f13e-4ae1-96f0-a02d369230f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:08.630719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25df13a-f13e-4ae1-96f0-a02d369230f4 None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:08.632059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25df13a-f13e-4ae1-96f0-a02d369230f4 None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:08.634335 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25df13a-f13e-4ae1-96f0-a02d369230f4 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:08.636519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 223/434] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:08 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:08.642593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25df13a-f13e-4ae1-96f0-a02d369230f4 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:08.708780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:08.718053 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d29bf-5fa1-499e-8e2e-bbe2b40d5bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:08.720231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d29bf-5fa1-499e-8e2e-bbe2b40d5bae None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:08.775796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25df13a-f13e-4ae1-96f0-a02d369230f4 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:08.812092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b5570-471a-4dd9-9212-af41fd385e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:08.812750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b5570-471a-4dd9-9212-af41fd385e8f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:08.813065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b5570-471a-4dd9-9212-af41fd385e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:08.813343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b5570-471a-4dd9-9212-af41fd385e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:08.814173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b5570-471a-4dd9-9212-af41fd385e8f None admin] RBAC: Adding query filter params (group.id=b6c303a21aec4dc1b49b7cc15d2eb063, role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=84b308b5ae9a4e0482f37715f327cd20) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:08.814485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b5570-471a-4dd9-9212-af41fd385e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:08.816320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b5570-471a-4dd9-9212-af41fd385e8f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:08.823569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 224/435] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:16:08 2022] GET /identity/v3/role_assignments?group.id=b6c303a21aec4dc1b49b7cc15d2eb063&scope.project.id=84b308b5ae9a4e0482f37715f327cd20&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:08.824796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad7296-fc29-4943-87c4-0e782b5780e1 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:08.825116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad7296-fc29-4943-87c4-0e782b5780e1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:08.825391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad7296-fc29-4943-87c4-0e782b5780e1 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:08.826828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 225/436] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:08.832523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00e3c-9d88-43a6-b839-2509f9352503 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:08.832829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00e3c-9d88-43a6-b839-2509f9352503 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:08.833120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00e3c-9d88-43a6-b839-2509f9352503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:08.851747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a2f00e3c-9d88-43a6-b839-2509f9352503 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:08.853258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a2f00e3c-9d88-43a6-b839-2509f9352503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:08.857694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:08.878069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 212/437] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:08.879334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d29bf-5fa1-499e-8e2e-bbe2b40d5bae None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:08.958990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 226/438] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:08.985529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00e3c-9d88-43a6-b839-2509f9352503 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:08.995858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:09.005147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d29bf-5fa1-499e-8e2e-bbe2b40d5bae None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:09.052037 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445ead-2eea-4303-ae1c-81c5c49bd4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:09.052925 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445ead-2eea-4303-ae1c-81c5c49bd4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.053209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445ead-2eea-4303-ae1c-81c5c49bd4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.053480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445ead-2eea-4303-ae1c-81c5c49bd4c7 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.057291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd445ead-2eea-4303-ae1c-81c5c49bd4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:09.059363 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd445ead-2eea-4303-ae1c-81c5c49bd4c7 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:09.061672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 213/439] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:08 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:09.067718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445ead-2eea-4303-ae1c-81c5c49bd4c7 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:09.125308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00e3c-9d88-43a6-b839-2509f9352503 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:09.168297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ca256-97e1-426a-9205-63994a0a2651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:09.169607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ca256-97e1-426a-9205-63994a0a2651 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.170107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ca256-97e1-426a-9205-63994a0a2651 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.170387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ca256-97e1-426a-9205-63994a0a2651 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.179099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:09.183954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ca256-97e1-426a-9205-63994a0a2651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-749ca256-97e1-426a-9205-63994a0a2651 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:09.192540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:09.193473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:09.194267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:09.194882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 227/440] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:16:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:09.200308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ca256-97e1-426a-9205-63994a0a2651 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:09.206930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445ead-2eea-4303-ae1c-81c5c49bd4c7 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:09.252755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185734a-596e-4d2d-9740-0170eab8c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:09.253711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185734a-596e-4d2d-9740-0170eab8c718 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.254095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185734a-596e-4d2d-9740-0170eab8c718 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.254471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185734a-596e-4d2d-9740-0170eab8c718 None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.255796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185734a-596e-4d2d-9740-0170eab8c718 None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:09.257536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185734a-596e-4d2d-9740-0170eab8c718 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:09.260032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 214/441] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:09 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:09.265641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185734a-596e-4d2d-9740-0170eab8c718 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:09.307224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:09.334150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ca256-97e1-426a-9205-63994a0a2651 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:09.379090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a3ba2-34c4-4f17-b46f-de4cc995aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:09.379770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a3ba2-34c4-4f17-b46f-de4cc995aa20 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.380046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a3ba2-34c4-4f17-b46f-de4cc995aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.380327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a3ba2-34c4-4f17-b46f-de4cc995aa20 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.381140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a3ba2-34c4-4f17-b46f-de4cc995aa20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:09.381453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a3ba2-34c4-4f17-b46f-de4cc995aa20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:09.383641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a3ba2-34c4-4f17-b46f-de4cc995aa20 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:09.393251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 228/442] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:09.398411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a3ba2-34c4-4f17-b46f-de4cc995aa20 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:09.402736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185734a-596e-4d2d-9740-0170eab8c718 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:09.438394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f898d0f-8adf-41bb-84f4-9874f66a874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:09.439050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f898d0f-8adf-41bb-84f4-9874f66a874a None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.439331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f898d0f-8adf-41bb-84f4-9874f66a874a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.439600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f898d0f-8adf-41bb-84f4-9874f66a874a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.440429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f898d0f-8adf-41bb-84f4-9874f66a874a None admin] RBAC: Adding query filter params (group.id=b6c303a21aec4dc1b49b7cc15d2eb063, role.id=8de5789832e84f9388b1295d479fca6e, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:09.440734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f898d0f-8adf-41bb-84f4-9874f66a874a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:09.442200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f898d0f-8adf-41bb-84f4-9874f66a874a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:09.448916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 215/443] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:16:09 2022] GET /identity/v3/role_assignments?group.id=b6c303a21aec4dc1b49b7cc15d2eb063&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=8de5789832e84f9388b1295d479fca6e => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:09.464891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:09.514797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a3ba2-34c4-4f17-b46f-de4cc995aa20 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:09.552041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c35f7-fe28-40b1-a68c-870a830da7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:09.553079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c35f7-fe28-40b1-a68c-870a830da7ec None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.553546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c35f7-fe28-40b1-a68c-870a830da7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.554006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c35f7-fe28-40b1-a68c-870a830da7ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.556967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af307c-fa7b-4a56-8543-334fb2cc490e None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.557255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af307c-fa7b-4a56-8543-334fb2cc490e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.557532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af307c-fa7b-4a56-8543-334fb2cc490e None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.559191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 216/444] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:09.564743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5bb3b-a730-475f-8a01-a67d3dbe52b9 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.565069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5bb3b-a730-475f-8a01-a67d3dbe52b9 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.565345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5bb3b-a730-475f-8a01-a67d3dbe52b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.583285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-71b5bb3b-a730-475f-8a01-a67d3dbe52b9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:09.584529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-71b5bb3b-a730-475f-8a01-a67d3dbe52b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:09.648834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c35f7-fe28-40b1-a68c-870a830da7ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:09.650373 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c35f7-fe28-40b1-a68c-870a830da7ec None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:09.671830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 217/445] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:09.696111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5bb3b-a730-475f-8a01-a67d3dbe52b9 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:09.718376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:09.730685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:09.765270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 229/446] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:16:09.772914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c35f7-fe28-40b1-a68c-870a830da7ec None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:09.815467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5bb3b-a730-475f-8a01-a67d3dbe52b9 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:09.858632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950735d-dd29-4938-b627-d4075a03f897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:09.859531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950735d-dd29-4938-b627-d4075a03f897 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.859820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950735d-dd29-4938-b627-d4075a03f897 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.860100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950735d-dd29-4938-b627-d4075a03f897 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.871281 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950735d-dd29-4938-b627-d4075a03f897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-0950735d-dd29-4938-b627-d4075a03f897 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:09.878992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:09.879890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:09.880884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 218/447] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:09.886111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950735d-dd29-4938-b627-d4075a03f897 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:09.893464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c35f7-fe28-40b1-a68c-870a830da7ec None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:09.926371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a043d6-5c41-4723-a251-218aeec0f833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:09.927223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a043d6-5c41-4723-a251-218aeec0f833 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:09.927499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a043d6-5c41-4723-a251-218aeec0f833 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:09.927769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a043d6-5c41-4723-a251-218aeec0f833 None admin] PATH_INFO: `/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:09.928954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a043d6-5c41-4723-a251-218aeec0f833 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cbcf4de61f4fb59bbc787e33742b7e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:09.931150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a043d6-5c41-4723-a251-218aeec0f833 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:09.932805 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 230/448] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:09 2022] GET /identity/v3/projects/b9cbcf4de61f4fb59bbc787e33742b7e => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:09.938099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a043d6-5c41-4723-a251-218aeec0f833 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:09.966880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:09.970545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:09.986205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950735d-dd29-4938-b627-d4075a03f897 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:10.020058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955cb9-cb5d-4286-bf50-4d835a2e3c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:10.020721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955cb9-cb5d-4286-bf50-4d835a2e3c75 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:10.021035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955cb9-cb5d-4286-bf50-4d835a2e3c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:10.021312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955cb9-cb5d-4286-bf50-4d835a2e3c75 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:10.022057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955cb9-cb5d-4286-bf50-4d835a2e3c75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:10.022357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955cb9-cb5d-4286-bf50-4d835a2e3c75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:10.023898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955cb9-cb5d-4286-bf50-4d835a2e3c75 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:10.031798 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 219/449] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:10.035456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a043d6-5c41-4723-a251-218aeec0f833 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:10.037205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955cb9-cb5d-4286-bf50-4d835a2e3c75 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:10.069752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85489-2a6b-4ae8-bd30-1942964c9d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:10.070868 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85489-2a6b-4ae8-bd30-1942964c9d27 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:10.071147 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85489-2a6b-4ae8-bd30-1942964c9d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:10.071424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85489-2a6b-4ae8-bd30-1942964c9d27 None admin] PATH_INFO: `/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:10.072651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d85489-2a6b-4ae8-bd30-1942964c9d27 None admin] RBAC: Authorizing `identity:get_group(group_id=b6c303a21aec4dc1b49b7cc15d2eb063)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:10.074169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d85489-2a6b-4ae8-bd30-1942964c9d27 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:10.080464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 231/450] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:09 2022] GET /identity/v3/groups/b6c303a21aec4dc1b49b7cc15d2eb063 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:10.081416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85489-2a6b-4ae8-bd30-1942964c9d27 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:10.138144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955cb9-cb5d-4286-bf50-4d835a2e3c75 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:10.172285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f74380-047c-4cd0-8f1b-e50f0adba78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:10.172960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f74380-047c-4cd0-8f1b-e50f0adba78c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:10.173245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f74380-047c-4cd0-8f1b-e50f0adba78c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:10.173524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f74380-047c-4cd0-8f1b-e50f0adba78c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:10.178461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:10.182791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85489-2a6b-4ae8-bd30-1942964c9d27 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:10.192452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:10.218442 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcf0ba-bb93-4edd-b893-620b2f8b20e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:10.219091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcf0ba-bb93-4edd-b893-620b2f8b20e7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:10.219374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcf0ba-bb93-4edd-b893-620b2f8b20e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:10.219651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcf0ba-bb93-4edd-b893-620b2f8b20e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:10.220487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcf0ba-bb93-4edd-b893-620b2f8b20e7 None admin] RBAC: Adding query filter params (group.id=b6c303a21aec4dc1b49b7cc15d2eb063, role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:10.220796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcf0ba-bb93-4edd-b893-620b2f8b20e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:10.222324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcf0ba-bb93-4edd-b893-620b2f8b20e7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:10.229052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 232/451] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:16:10 2022] GET /identity/v3/role_assignments?group.id=b6c303a21aec4dc1b49b7cc15d2eb063&scope.project.id=b9cbcf4de61f4fb59bbc787e33742b7e&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:10.247848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f74380-047c-4cd0-8f1b-e50f0adba78c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:10.249412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f74380-047c-4cd0-8f1b-e50f0adba78c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:10.358126 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 220/452] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:16:10.363926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcf0ba-bb93-4edd-b893-620b2f8b20e7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:10.467582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcf0ba-bb93-4edd-b893-620b2f8b20e7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:10.501188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca133dc-2dec-4e02-9032-21e522006636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:10.502025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca133dc-2dec-4e02-9032-21e522006636 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:10.502304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca133dc-2dec-4e02-9032-21e522006636 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:10.502577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca133dc-2dec-4e02-9032-21e522006636 None admin] PATH_INFO: `/v3/projects/c586cdfef8dc4cea9cf389817d345b08` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:10.503736 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca133dc-2dec-4e02-9032-21e522006636 None admin] RBAC: Authorizing `identity:get_project(project_id=c586cdfef8dc4cea9cf389817d345b08)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:10.505186 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca133dc-2dec-4e02-9032-21e522006636 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:10.506847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 233/453] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:16:10 2022] GET /identity/v3/projects/c586cdfef8dc4cea9cf389817d345b08 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:10.512847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f74380-047c-4cd0-8f1b-e50f0adba78c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:10.564314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:10.638961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f74380-047c-4cd0-8f1b-e50f0adba78c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:10.672416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72fc2a-8cd9-4e61-91ec-a2e6d8d8b9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:10.673295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72fc2a-8cd9-4e61-91ec-a2e6d8d8b9aa None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:10.673585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72fc2a-8cd9-4e61-91ec-a2e6d8d8b9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:10.673860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72fc2a-8cd9-4e61-91ec-a2e6d8d8b9aa None admin] PATH_INFO: `/v3/groups/f3eb814f1aad49369d340c6264d6c39a` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:10.675012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72fc2a-8cd9-4e61-91ec-a2e6d8d8b9aa None admin] RBAC: Authorizing `identity:get_group(group_id=f3eb814f1aad49369d340c6264d6c39a)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:10.676462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72fc2a-8cd9-4e61-91ec-a2e6d8d8b9aa None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:10.678101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 221/454] 10.222.0.30 () {56 vars in 1180 bytes} [Mon May 23 15:16:10 2022] GET /identity/v3/groups/f3eb814f1aad49369d340c6264d6c39a => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:10.684140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca133dc-2dec-4e02-9032-21e522006636 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:10.714585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:10.781518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca133dc-2dec-4e02-9032-21e522006636 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:10.814706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45f445-ef2d-4b58-b36b-f6520cc6643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:10.815349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f445-ef2d-4b58-b36b-f6520cc6643c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:10.815631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f445-ef2d-4b58-b36b-f6520cc6643c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:10.815907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f445-ef2d-4b58-b36b-f6520cc6643c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:10.816728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45f445-ef2d-4b58-b36b-f6520cc6643c None admin] RBAC: Adding query filter params (group.id=f3eb814f1aad49369d340c6264d6c39a, role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=c586cdfef8dc4cea9cf389817d345b08) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:10.817065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45f445-ef2d-4b58-b36b-f6520cc6643c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:10.818503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45f445-ef2d-4b58-b36b-f6520cc6643c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:10.824788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 234/455] 10.222.0.30 () {56 vars in 1376 bytes} [Mon May 23 15:16:10 2022] GET /identity/v3/role_assignments?group.id=f3eb814f1aad49369d340c6264d6c39a&scope.project.id=c586cdfef8dc4cea9cf389817d345b08&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:13.474964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ee43-5800-4c53-9f7c-db967d70dac7 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:13.475511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ee43-5800-4c53-9f7c-db967d70dac7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:13.475587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ee43-5800-4c53-9f7c-db967d70dac7 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:13.478275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 222/456] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:13.483711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c68201-3d48-4a7b-887b-5571f1870cbf None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:13.484652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c68201-3d48-4a7b-887b-5571f1870cbf None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:13.485156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c68201-3d48-4a7b-887b-5571f1870cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:13.502988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d2c68201-3d48-4a7b-887b-5571f1870cbf None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:13.504511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d2c68201-3d48-4a7b-887b-5571f1870cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:13.616531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 235/457] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:13.653791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ee43-5800-4c53-9f7c-db967d70dac7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:13.683437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5411011-aa6b-42d9-8ece-4329aa181618 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:13.684101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5411011-aa6b-42d9-8ece-4329aa181618 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:13.684598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5411011-aa6b-42d9-8ece-4329aa181618 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:13.686194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 236/458] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:13.697666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d9680-4350-4e0c-9f44-16583622c06e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:13.698103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d9680-4350-4e0c-9f44-16583622c06e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:13.698474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d9680-4350-4e0c-9f44-16583622c06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:13.718077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-5f5d9680-4350-4e0c-9f44-16583622c06e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:13.720240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5f5d9680-4350-4e0c-9f44-16583622c06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:13.740217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:13.840575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 237/459] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:13.841883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ee43-5800-4c53-9f7c-db967d70dac7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:13.864466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d9680-4350-4e0c-9f44-16583622c06e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:13.876281 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:13.909391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39f520-e135-4198-8d38-9e164f9dfad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:13.910191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39f520-e135-4198-8d38-9e164f9dfad6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:13.910474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39f520-e135-4198-8d38-9e164f9dfad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:13.910752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39f520-e135-4198-8d38-9e164f9dfad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:13.932717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39f520-e135-4198-8d38-9e164f9dfad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:13.934408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39f520-e135-4198-8d38-9e164f9dfad6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-9f39f520-e135-4198-8d38-9e164f9dfad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:13.965513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:13.966654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:13.967565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:13.967565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:13.967565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:13.967565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:13.967956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 223/460] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:13.978059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39f520-e135-4198-8d38-9e164f9dfad6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:14.031944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d9680-4350-4e0c-9f44-16583622c06e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:14.078274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694dd5-b573-49e7-8e34-fc42c880745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:14.079120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694dd5-b573-49e7-8e34-fc42c880745e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.079399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694dd5-b573-49e7-8e34-fc42c880745e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.079671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694dd5-b573-49e7-8e34-fc42c880745e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.091587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae694dd5-b573-49e7-8e34-fc42c880745e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:14.093094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae694dd5-b573-49e7-8e34-fc42c880745e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:14.110688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39f520-e135-4198-8d38-9e164f9dfad6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-ae694dd5-b573-49e7-8e34-fc42c880745e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:14.111830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:14.112640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:14.113513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:14.113513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:14.113513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:14.113513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:14.113513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 238/461] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:14.118645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71478f-a71d-455d-be79-47da15035e3c None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.118645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71478f-a71d-455d-be79-47da15035e3c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.118645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71478f-a71d-455d-be79-47da15035e3c None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.121299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 239/462] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:14.122119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71478f-a71d-455d-be79-47da15035e3c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:14.131192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:14.158683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf6a14-423a-4aa6-bc18-e402d458fe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:14.159337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6a14-423a-4aa6-bc18-e402d458fe4d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.159614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6a14-423a-4aa6-bc18-e402d458fe4d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.159883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6a14-423a-4aa6-bc18-e402d458fe4d None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.160608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf6a14-423a-4aa6-bc18-e402d458fe4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:14.160930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf6a14-423a-4aa6-bc18-e402d458fe4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:14.162321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf6a14-423a-4aa6-bc18-e402d458fe4d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:14.181964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 224/463] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:14.183376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bca01b-8066-43c6-a9ec-fc5bab98ab98 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.183754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bca01b-8066-43c6-a9ec-fc5bab98ab98 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.183949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bca01b-8066-43c6-a9ec-fc5bab98ab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.201944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e4bca01b-8066-43c6-a9ec-fc5bab98ab98 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:14.203268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e4bca01b-8066-43c6-a9ec-fc5bab98ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:14.228455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:14.255253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71478f-a71d-455d-be79-47da15035e3c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:14.278498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 225/464] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:14.279279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bca01b-8066-43c6-a9ec-fc5bab98ab98 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:14.303816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c559de-5cf5-4bf5-9d59-c45486ac8892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:14.304906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c559de-5cf5-4bf5-9d59-c45486ac8892 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.305308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c559de-5cf5-4bf5-9d59-c45486ac8892 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.305722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c559de-5cf5-4bf5-9d59-c45486ac8892 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.306698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c559de-5cf5-4bf5-9d59-c45486ac8892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:14.307010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c559de-5cf5-4bf5-9d59-c45486ac8892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:14.309078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c559de-5cf5-4bf5-9d59-c45486ac8892 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:14.324096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 240/465] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:14.328018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05801f-58c3-4bf1-9db7-54302cf639a1 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.328018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05801f-58c3-4bf1-9db7-54302cf639a1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.328018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05801f-58c3-4bf1-9db7-54302cf639a1 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.328018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 241/466] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:14.328018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05801f-58c3-4bf1-9db7-54302cf639a1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:14.397670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bca01b-8066-43c6-a9ec-fc5bab98ab98 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:14.406379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:14.436000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c4e66-287c-4b14-8988-659615209a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:14.437156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c4e66-287c-4b14-8988-659615209a85 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.437614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c4e66-287c-4b14-8988-659615209a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.438070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c4e66-287c-4b14-8988-659615209a85 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.439069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c4e66-287c-4b14-8988-659615209a85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:14.440794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c4e66-287c-4b14-8988-659615209a85 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:14.442888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05801f-58c3-4bf1-9db7-54302cf639a1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:14.463275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 226/467] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:16:14 2022] POST /identity/v3/users => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:16:14.464318 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c4e66-287c-4b14-8988-659615209a85 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:14.487117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a58d3-3dc2-41a7-b3e9-25c4d8f1f67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:14.488030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a58d3-3dc2-41a7-b3e9-25c4d8f1f67a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.488543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a58d3-3dc2-41a7-b3e9-25c4d8f1f67a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.489081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a58d3-3dc2-41a7-b3e9-25c4d8f1f67a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.497289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:14.504926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a58d3-3dc2-41a7-b3e9-25c4d8f1f67a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:14.506973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a58d3-3dc2-41a7-b3e9-25c4d8f1f67a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-853a58d3-3dc2-41a7-b3e9-25c4d8f1f67a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:14.527864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:14.528767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:14.529646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:14.529646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:14.529646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:14.529646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:14.532072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 242/468] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:14.533965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ac14-b359-40ec-a4ea-4112dc42ce12 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.534508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ac14-b359-40ec-a4ea-4112dc42ce12 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.534983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ac14-b359-40ec-a4ea-4112dc42ce12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.554030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1a62ac14-b359-40ec-a4ea-4112dc42ce12 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:14.556391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1a62ac14-b359-40ec-a4ea-4112dc42ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:14.599716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c4e66-287c-4b14-8988-659615209a85 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:14.641615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4402d8d-237a-405a-8293-78d13cf5d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:14.642289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4402d8d-237a-405a-8293-78d13cf5d466 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.642687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4402d8d-237a-405a-8293-78d13cf5d466 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.642860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4402d8d-237a-405a-8293-78d13cf5d466 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.643551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4402d8d-237a-405a-8293-78d13cf5d466 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:14.645126 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4402d8d-237a-405a-8293-78d13cf5d466 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:14.656959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 243/469] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:14.661825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ac14-b359-40ec-a4ea-4112dc42ce12 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:14.666299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 227/470] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:16:14 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:16:14.692638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4402d8d-237a-405a-8293-78d13cf5d466 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:14.736811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:14.788488 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:14.817977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ac14-b359-40ec-a4ea-4112dc42ce12 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:14.819223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4402d8d-237a-405a-8293-78d13cf5d466 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:14.863463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c33c4-87e6-4d2f-b264-8aa04ab2a5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:14.864277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c33c4-87e6-4d2f-b264-8aa04ab2a5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.864554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c33c4-87e6-4d2f-b264-8aa04ab2a5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.864830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c33c4-87e6-4d2f-b264-8aa04ab2a5d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.867612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495b415-3aff-4bbd-8467-7feac13dbf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:14.868604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495b415-3aff-4bbd-8467-7feac13dbf5b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:14.869159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495b415-3aff-4bbd-8467-7feac13dbf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:14.869626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495b415-3aff-4bbd-8467-7feac13dbf5b None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:14.870557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3495b415-3aff-4bbd-8467-7feac13dbf5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:14.871313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3495b415-3aff-4bbd-8467-7feac13dbf5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:14.872899 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3495b415-3aff-4bbd-8467-7feac13dbf5b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:14.877512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c33c4-87e6-4d2f-b264-8aa04ab2a5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:14.879081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c33c4-87e6-4d2f-b264-8aa04ab2a5d3 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:14.887402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 244/471] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:14.892775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495b415-3aff-4bbd-8467-7feac13dbf5b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-940c33c4-87e6-4d2f-b264-8aa04ab2a5d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:14.897928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:14.898791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:14.899632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:14.899632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:14.899632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:14.899632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:14.899963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 228/472] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:14.905335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c33c4-87e6-4d2f-b264-8aa04ab2a5d3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:15.017265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495b415-3aff-4bbd-8467-7feac13dbf5b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:15.028627 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:15.033728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c33c4-87e6-4d2f-b264-8aa04ab2a5d3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:15.060655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94408da8-45b2-4175-a146-37bc2690a2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:15.061367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408da8-45b2-4175-a146-37bc2690a2c7 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:15.061816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408da8-45b2-4175-a146-37bc2690a2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:15.062189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408da8-45b2-4175-a146-37bc2690a2c7 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:15.063659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94408da8-45b2-4175-a146-37bc2690a2c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:15.065298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94408da8-45b2-4175-a146-37bc2690a2c7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:15.069278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:15.078005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401e3de-c746-479e-b15a-aa69aff30ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:15.079714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401e3de-c746-479e-b15a-aa69aff30ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:15.080325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401e3de-c746-479e-b15a-aa69aff30ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:15.081482 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401e3de-c746-479e-b15a-aa69aff30ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:15.082762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401e3de-c746-479e-b15a-aa69aff30ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:15.083223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401e3de-c746-479e-b15a-aa69aff30ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:15.086588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401e3de-c746-479e-b15a-aa69aff30ea8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:15.096090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 245/473] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:16:14 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:16:15.097808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f036d1-85d4-46a8-8e3d-02954233c592 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:15.098294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f036d1-85d4-46a8-8e3d-02954233c592 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:15.098730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f036d1-85d4-46a8-8e3d-02954233c592 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:15.100390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 246/474] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:15.101918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 229/475] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:15.106534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e23405-1746-4108-a2d9-bdac1cfb7fe0 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:15.106994 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e23405-1746-4108-a2d9-bdac1cfb7fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:15.107373 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e23405-1746-4108-a2d9-bdac1cfb7fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:15.107558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401e3de-c746-479e-b15a-aa69aff30ea8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:15.126327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-40e23405-1746-4108-a2d9-bdac1cfb7fe0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:15.127616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-40e23405-1746-4108-a2d9-bdac1cfb7fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:15.204921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 247/476] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:15.228678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e23405-1746-4108-a2d9-bdac1cfb7fe0 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:15.230311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401e3de-c746-479e-b15a-aa69aff30ea8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:15.277014 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e10406-565c-4f15-92c4-67dd83483306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:15.278272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e10406-565c-4f15-92c4-67dd83483306 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:15.278763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e10406-565c-4f15-92c4-67dd83483306 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:15.279335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e10406-565c-4f15-92c4-67dd83483306 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:15.280259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e10406-565c-4f15-92c4-67dd83483306 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:15.282222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e10406-565c-4f15-92c4-67dd83483306 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:15.320059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 230/477] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:16:15 2022] POST /identity/v3/users => generated 249 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:16:15.346429 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:15.361141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e23405-1746-4108-a2d9-bdac1cfb7fe0 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:15.404023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:15.404762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:15.405084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:15.405367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:15.406187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:15.407620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:15.415161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.sql.core [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 15:16:15.415161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 15:16:15.415161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [parameters: {'id': 'c137132295484f01b4ad3cb12debb06c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 15:16:15.415161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80422) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application conn.query(q) May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application result.read() May 23 15:16:15.435052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return ref.to_dict() May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:16:15.435985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application next(self.gen) May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application yield resource May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application next(self.gen) May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.session.rollback() May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.force_reraise() May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise self.value May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application session.commit() May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 15:16:15.437411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._prepare_impl() May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.session.flush() May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._flush(objects) May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application compat.raise_( May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application flush_context.execute() May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rec.execute(self) May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 15:16:15.439248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:16:15.441315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application conn.query(q) May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application result.read() May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:16:15.445872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [parameters: {'id': 'c137132295484f01b4ad3cb12debb06c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:15.448960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 15:16:15.450977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:15.457704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 248/478] 10.222.0.30 () {60 vars in 1131 bytes} [Mon May 23 15:16:15 2022] POST /identity/v3/roles => generated 144 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:15.457821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e10406-565c-4f15-92c4-67dd83483306 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:15.502288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:15.579404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e10406-565c-4f15-92c4-67dd83483306 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:15.619488 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f26b0-ef27-4965-a300-270fde44e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:15.620304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f26b0-ef27-4965-a300-270fde44e3ce None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:15.620584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f26b0-ef27-4965-a300-270fde44e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:15.621195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f26b0-ef27-4965-a300-270fde44e3ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:15.635237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f26b0-ef27-4965-a300-270fde44e3ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-8f0f26b0-ef27-4965-a300-270fde44e3ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:15.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:15.644338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:15.645179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 231/479] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:16:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:15.651587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:15.771344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bc3fb-f53e-4ba8-85ce-e7a048b6be07 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:15.812725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd99e0-f653-45f8-b7db-d29088deb0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:15.813436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd99e0-f653-45f8-b7db-d29088deb0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:15.813718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd99e0-f653-45f8-b7db-d29088deb0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:15.813990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd99e0-f653-45f8-b7db-d29088deb0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:15.814744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd99e0-f653-45f8-b7db-d29088deb0a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:15.815044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd99e0-f653-45f8-b7db-d29088deb0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:15.816408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd99e0-f653-45f8-b7db-d29088deb0a0 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:15.825843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 249/480] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:17.308006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931d927-f505-4acc-9029-feb3a5dd2e9e None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:17.308550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931d927-f505-4acc-9029-feb3a5dd2e9e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:17.308550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931d927-f505-4acc-9029-feb3a5dd2e9e None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:17.310306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 232/481] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:17.323785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8a30-9ffb-4f48-a89a-b9fb1e6062a1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:17.324220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8a30-9ffb-4f48-a89a-b9fb1e6062a1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:17.324393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8a30-9ffb-4f48-a89a-b9fb1e6062a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:17.345852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4c2b8a30-9ffb-4f48-a89a-b9fb1e6062a1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:17.347045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4c2b8a30-9ffb-4f48-a89a-b9fb1e6062a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:17.363082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:17.439481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 250/482] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:17.482701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931d927-f505-4acc-9029-feb3a5dd2e9e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:17.491980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa3d2e-f802-43cb-8009-b329c175cbc8 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:17.492721 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa3d2e-f802-43cb-8009-b329c175cbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:17.493043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa3d2e-f802-43cb-8009-b329c175cbc8 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:17.500388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 251/483] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:17.507537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f1b7d-58d5-407f-9f4c-c1f8c0664313 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:17.507789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f1b7d-58d5-407f-9f4c-c1f8c0664313 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:17.508067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f1b7d-58d5-407f-9f4c-c1f8c0664313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:17.536932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4a4f1b7d-58d5-407f-9f4c-c1f8c0664313 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:17.537954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4a4f1b7d-58d5-407f-9f4c-c1f8c0664313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:17.644977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 252/484] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:17.662129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f1b7d-58d5-407f-9f4c-c1f8c0664313 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:17.718344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:17.728492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931d927-f505-4acc-9029-feb3a5dd2e9e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:17.747081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:17.767492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f1b7d-58d5-407f-9f4c-c1f8c0664313 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:17.802009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac39888-f004-4ff0-986e-c95db53f7516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:17.802807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac39888-f004-4ff0-986e-c95db53f7516 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:17.803026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479e1b2-588c-4bd3-a118-63ba46fbdfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:17.803166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac39888-f004-4ff0-986e-c95db53f7516 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:17.803343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac39888-f004-4ff0-986e-c95db53f7516 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:17.803826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479e1b2-588c-4bd3-a118-63ba46fbdfee None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:17.804108 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479e1b2-588c-4bd3-a118-63ba46fbdfee None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:17.804957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479e1b2-588c-4bd3-a118-63ba46fbdfee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:17.815082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac39888-f004-4ff0-986e-c95db53f7516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-cac39888-f004-4ff0-986e-c95db53f7516 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:17.823611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:17.824660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:17.825733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:17.832276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479e1b2-588c-4bd3-a118-63ba46fbdfee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:17.832513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 253/485] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:17.833856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac39888-f004-4ff0-986e-c95db53f7516 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-9479e1b2-588c-4bd3-a118-63ba46fbdfee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:17.846082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:17.847156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:17.848173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 233/486] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:17.854045 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479e1b2-588c-4bd3-a118-63ba46fbdfee None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:17.972269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac39888-f004-4ff0-986e-c95db53f7516 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:18.044969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e761c5c-0dd1-4f88-af42-69a06b16f51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:18.046949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e761c5c-0dd1-4f88-af42-69a06b16f51c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.047372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e761c5c-0dd1-4f88-af42-69a06b16f51c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.048465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e761c5c-0dd1-4f88-af42-69a06b16f51c None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.049418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e761c5c-0dd1-4f88-af42-69a06b16f51c None admin] RBAC: Adding query filter params (name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:18.050471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e761c5c-0dd1-4f88-af42-69a06b16f51c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:18.052124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e761c5c-0dd1-4f88-af42-69a06b16f51c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:18.059351 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479e1b2-588c-4bd3-a118-63ba46fbdfee None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:18.072054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 254/487] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:18.073678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e761c5c-0dd1-4f88-af42-69a06b16f51c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:18.093273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:18.138119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e5ddf-e47f-4b42-8847-4e6a69c81fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:18.139034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e5ddf-e47f-4b42-8847-4e6a69c81fee None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.139474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e5ddf-e47f-4b42-8847-4e6a69c81fee None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.139899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e5ddf-e47f-4b42-8847-4e6a69c81fee None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.140816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e5ddf-e47f-4b42-8847-4e6a69c81fee None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:18.141376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e5ddf-e47f-4b42-8847-4e6a69c81fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:18.143022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e5ddf-e47f-4b42-8847-4e6a69c81fee None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:18.164919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 234/488] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:18.170015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e5ddf-e47f-4b42-8847-4e6a69c81fee None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:18.212157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:18.259711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e761c5c-0dd1-4f88-af42-69a06b16f51c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:18.304274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e62489-f122-4106-a38e-2d250597ed99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:18.306077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e62489-f122-4106-a38e-2d250597ed99 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.306503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e62489-f122-4106-a38e-2d250597ed99 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.307501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e62489-f122-4106-a38e-2d250597ed99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.318628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e5ddf-e47f-4b42-8847-4e6a69c81fee None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:18.322876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e62489-f122-4106-a38e-2d250597ed99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:18.323863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e62489-f122-4106-a38e-2d250597ed99 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-a8e62489-f122-4106-a38e-2d250597ed99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:18.345783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:18.346681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:18.347490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:18.347490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:18.347490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:18.347490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:18.347490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 255/489] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:18.348582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65422f-62c7-49bd-84f0-4808b6ce42a0 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.348879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65422f-62c7-49bd-84f0-4808b6ce42a0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.349153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65422f-62c7-49bd-84f0-4808b6ce42a0 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.354693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 256/490] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:18.354693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbea9b-4aeb-4ae4-984a-dac20c676431 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.354693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbea9b-4aeb-4ae4-984a-dac20c676431 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.354693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbea9b-4aeb-4ae4-984a-dac20c676431 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.354693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 257/491] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:18.355045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbea9b-4aeb-4ae4-984a-dac20c676431 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:18.371661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb9eb6-71b6-4d04-81d1-36c7d58bf78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:18.372453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb9eb6-71b6-4d04-81d1-36c7d58bf78c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.372732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb9eb6-71b6-4d04-81d1-36c7d58bf78c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.373030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb9eb6-71b6-4d04-81d1-36c7d58bf78c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.385254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb9eb6-71b6-4d04-81d1-36c7d58bf78c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:18.386853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb9eb6-71b6-4d04-81d1-36c7d58bf78c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-72bb9eb6-71b6-4d04-81d1-36c7d58bf78c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:18.404355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:18.405256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:18.406056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:18.406056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:18.406056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:18.406056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:18.406056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 235/492] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:18.406884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092c50-7aaf-4646-9ecf-c01f9efe4b45 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.407104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092c50-7aaf-4646-9ecf-c01f9efe4b45 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.407380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092c50-7aaf-4646-9ecf-c01f9efe4b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.422282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:18.424950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-42092c50-7aaf-4646-9ecf-c01f9efe4b45 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:18.426277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-42092c50-7aaf-4646-9ecf-c01f9efe4b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:18.471900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbea9b-4aeb-4ae4-984a-dac20c676431 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:18.500805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 236/493] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:18.502129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12911c-fdfd-404c-bf1f-cf6d9a7e602b None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.502419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12911c-fdfd-404c-bf1f-cf6d9a7e602b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.502698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12911c-fdfd-404c-bf1f-cf6d9a7e602b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.508573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:18.522567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8e12911c-fdfd-404c-bf1f-cf6d9a7e602b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:18.524397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8e12911c-fdfd-404c-bf1f-cf6d9a7e602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:18.536002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c5da1-784e-4a11-b4d6-61c72c94d306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:18.537267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c5da1-784e-4a11-b4d6-61c72c94d306 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.537771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c5da1-784e-4a11-b4d6-61c72c94d306 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.538503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c5da1-784e-4a11-b4d6-61c72c94d306 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.540071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c5da1-784e-4a11-b4d6-61c72c94d306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:18.540838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c5da1-784e-4a11-b4d6-61c72c94d306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:18.542689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c5da1-784e-4a11-b4d6-61c72c94d306 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:18.562525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 258/494] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:18.562525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c5da1-784e-4a11-b4d6-61c72c94d306 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:18.602341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 237/495] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:18.609903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12911c-fdfd-404c-bf1f-cf6d9a7e602b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:18.672312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c5da1-784e-4a11-b4d6-61c72c94d306 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:18.687513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:18.707650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37877ae4-0c95-43b1-a4e3-327521be250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:18.708306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37877ae4-0c95-43b1-a4e3-327521be250f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.708589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37877ae4-0c95-43b1-a4e3-327521be250f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.708859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37877ae4-0c95-43b1-a4e3-327521be250f None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.709593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37877ae4-0c95-43b1-a4e3-327521be250f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:18.709893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37877ae4-0c95-43b1-a4e3-327521be250f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:18.711269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37877ae4-0c95-43b1-a4e3-327521be250f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:18.720031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12911c-fdfd-404c-bf1f-cf6d9a7e602b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:18.722236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 259/496] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:18.723016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37877ae4-0c95-43b1-a4e3-327521be250f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:18.750041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:18.755152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436b363-2db6-4cd9-8f1e-423ad16d6651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:18.755942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b363-2db6-4cd9-8f1e-423ad16d6651 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.756219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b363-2db6-4cd9-8f1e-423ad16d6651 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.756491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b363-2db6-4cd9-8f1e-423ad16d6651 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.766619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436b363-2db6-4cd9-8f1e-423ad16d6651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-6436b363-2db6-4cd9-8f1e-423ad16d6651 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:18.774522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:18.775471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:18.776266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 238/497] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:18.777271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436b363-2db6-4cd9-8f1e-423ad16d6651 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:18.828971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37877ae4-0c95-43b1-a4e3-327521be250f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:18.862475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46964cb7-2448-41e7-9fe0-67513cfa674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:18.863261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46964cb7-2448-41e7-9fe0-67513cfa674c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.863559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46964cb7-2448-41e7-9fe0-67513cfa674c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.863827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46964cb7-2448-41e7-9fe0-67513cfa674c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.871981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46964cb7-2448-41e7-9fe0-67513cfa674c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:18.873437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46964cb7-2448-41e7-9fe0-67513cfa674c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:18.877092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436b363-2db6-4cd9-8f1e-423ad16d6651 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-46964cb7-2448-41e7-9fe0-67513cfa674c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:18.889192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:18.890196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:18.891081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 260/498] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:18.891624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bc3c4-61c9-4020-846a-59235a04e44f None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.891832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bc3c4-61c9-4020-846a-59235a04e44f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.892077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bc3c4-61c9-4020-846a-59235a04e44f None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.893348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 261/499] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:18.894035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bc3c4-61c9-4020-846a-59235a04e44f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:18.915828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e705c5-a365-48ce-97cb-7dbbd8f3d821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:18.916623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e705c5-a365-48ce-97cb-7dbbd8f3d821 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:18.916930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e705c5-a365-48ce-97cb-7dbbd8f3d821 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:18.917210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e705c5-a365-48ce-97cb-7dbbd8f3d821 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:18.927489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e705c5-a365-48ce-97cb-7dbbd8f3d821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:18.929537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-23e705c5-a365-48ce-97cb-7dbbd8f3d821 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:18.935884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:18.936984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 239/500] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:18.937980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e705c5-a365-48ce-97cb-7dbbd8f3d821 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:18.991149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:19.001784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bc3c4-61c9-4020-846a-59235a04e44f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.034399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0303f1-8b9f-4ed7-8572-bdf4a2fc3621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.035127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0303f1-8b9f-4ed7-8572-bdf4a2fc3621 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.035411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0303f1-8b9f-4ed7-8572-bdf4a2fc3621 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.035686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0303f1-8b9f-4ed7-8572-bdf4a2fc3621 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.038076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e705c5-a365-48ce-97cb-7dbbd8f3d821 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.043711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0303f1-8b9f-4ed7-8572-bdf4a2fc3621 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.045183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0303f1-8b9f-4ed7-8572-bdf4a2fc3621 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-3b0303f1-8b9f-4ed7-8572-bdf4a2fc3621 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:19.059800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:19.060696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 262/501] 10.222.0.30 () {56 vars in 1093 bytes} [Mon May 23 15:16:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:19.066350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0303f1-8b9f-4ed7-8572-bdf4a2fc3621 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.072618 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c644d-ab57-4855-938d-534b89f7bc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.073322 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c644d-ab57-4855-938d-534b89f7bc7c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.073602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c644d-ab57-4855-938d-534b89f7bc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.073872 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c644d-ab57-4855-938d-534b89f7bc7c None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.074624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c644d-ab57-4855-938d-534b89f7bc7c None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:19.074936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c644d-ab57-4855-938d-534b89f7bc7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.076305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c644d-ab57-4855-938d-534b89f7bc7c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.084466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 240/502] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:19.086381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62d36-dae8-4903-8221-ab637b0bf95e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.086790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62d36-dae8-4903-8221-ab637b0bf95e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.087152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62d36-dae8-4903-8221-ab637b0bf95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.103632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-cdc62d36-dae8-4903-8221-ab637b0bf95e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:19.104805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-cdc62d36-dae8-4903-8221-ab637b0bf95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:19.163440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:19.170047 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 241/503] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:19.170874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62d36-dae8-4903-8221-ab637b0bf95e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.171039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0303f1-8b9f-4ed7-8572-bdf4a2fc3621 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.211541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384bede-d3f7-40ad-a40a-6836ad30310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.212369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384bede-d3f7-40ad-a40a-6836ad30310c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.212647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384bede-d3f7-40ad-a40a-6836ad30310c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.212938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384bede-d3f7-40ad-a40a-6836ad30310c None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.213769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384bede-d3f7-40ad-a40a-6836ad30310c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:19.214071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384bede-d3f7-40ad-a40a-6836ad30310c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.215494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384bede-d3f7-40ad-a40a-6836ad30310c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.221182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:19.251873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 263/504] 10.222.0.30 () {56 vars in 1137 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:19.252688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384bede-d3f7-40ad-a40a-6836ad30310c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.275001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62d36-dae8-4903-8221-ab637b0bf95e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.308191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16a48f-a8ef-4e2f-8946-23dc7eb0d034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.308777 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16a48f-a8ef-4e2f-8946-23dc7eb0d034 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.309079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16a48f-a8ef-4e2f-8946-23dc7eb0d034 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.309348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16a48f-a8ef-4e2f-8946-23dc7eb0d034 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.310093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16a48f-a8ef-4e2f-8946-23dc7eb0d034 None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:19.310647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16a48f-a8ef-4e2f-8946-23dc7eb0d034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.311747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16a48f-a8ef-4e2f-8946-23dc7eb0d034 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.319457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 242/505] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:19.319457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16a48f-a8ef-4e2f-8946-23dc7eb0d034 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.353274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384bede-d3f7-40ad-a40a-6836ad30310c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.386289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5c1fc-a18b-49db-a56f-010dfcac910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.386950 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5c1fc-a18b-49db-a56f-010dfcac910e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.387228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5c1fc-a18b-49db-a56f-010dfcac910e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.387500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5c1fc-a18b-49db-a56f-010dfcac910e None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.388326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5c1fc-a18b-49db-a56f-010dfcac910e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:19.388631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5c1fc-a18b-49db-a56f-010dfcac910e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.390086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5c1fc-a18b-49db-a56f-010dfcac910e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.407518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 264/506] 10.222.0.30 () {56 vars in 1133 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:19.408332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5c1fc-a18b-49db-a56f-010dfcac910e None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.420886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:19.429958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16a48f-a8ef-4e2f-8946-23dc7eb0d034 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.437131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:19.466725 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bfcd5-81cd-4a83-884e-fa364249d8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.467514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bfcd5-81cd-4a83-884e-fa364249d8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.467791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bfcd5-81cd-4a83-884e-fa364249d8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.468067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bfcd5-81cd-4a83-884e-fa364249d8f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.479805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bfcd5-81cd-4a83-884e-fa364249d8f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.481315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bfcd5-81cd-4a83-884e-fa364249d8f0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-0b5bfcd5-81cd-4a83-884e-fa364249d8f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:19.499094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:19.499957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:19.500824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:19.500824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:19.500824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:19.500824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:19.500824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 243/507] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:19.501199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bfcd5-81cd-4a83-884e-fa364249d8f0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.525956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5c1fc-a18b-49db-a56f-010dfcac910e None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.559772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd601a3e-d7d6-4d17-91e0-48406a85fefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.560521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd601a3e-d7d6-4d17-91e0-48406a85fefd None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.560807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd601a3e-d7d6-4d17-91e0-48406a85fefd None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.561137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd601a3e-d7d6-4d17-91e0-48406a85fefd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.573259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd601a3e-d7d6-4d17-91e0-48406a85fefd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.574723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd601a3e-d7d6-4d17-91e0-48406a85fefd None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-cd601a3e-d7d6-4d17-91e0-48406a85fefd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:19.593193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:19.594234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:19.595208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:19.595208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:19.595208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:19.595208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:19.595208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 265/508] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:19.595567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd601a3e-d7d6-4d17-91e0-48406a85fefd None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.606283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bfcd5-81cd-4a83-884e-fa364249d8f0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.640460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c30bd-1f47-451f-8a9d-07537d462fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.641250 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c30bd-1f47-451f-8a9d-07537d462fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.641534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c30bd-1f47-451f-8a9d-07537d462fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.641816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c30bd-1f47-451f-8a9d-07537d462fc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.660415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:19.668837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:19.700712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd601a3e-d7d6-4d17-91e0-48406a85fefd None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.711457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c30bd-1f47-451f-8a9d-07537d462fc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.712991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c30bd-1f47-451f-8a9d-07537d462fc5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.733798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0be91-2682-49d0-8568-b5dbff8809de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.734581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0be91-2682-49d0-8568-b5dbff8809de None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.734860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0be91-2682-49d0-8568-b5dbff8809de None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.735128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0be91-2682-49d0-8568-b5dbff8809de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.747699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0be91-2682-49d0-8568-b5dbff8809de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.749165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0be91-2682-49d0-8568-b5dbff8809de None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-d8c0be91-2682-49d0-8568-b5dbff8809de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:19.766777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:19.767668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:19.768894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:19.768894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:19.768894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:19.768894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:19.768894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 266/509] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:19.768894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0be91-2682-49d0-8568-b5dbff8809de None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.822197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 244/510] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:19.823082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c30bd-1f47-451f-8a9d-07537d462fc5 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.881604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:19.885600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0be91-2682-49d0-8568-b5dbff8809de None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.899952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:19.921586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf852edd-357c-4fc9-a180-9a3e9c1e8700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.922257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf852edd-357c-4fc9-a180-9a3e9c1e8700 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.922542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf852edd-357c-4fc9-a180-9a3e9c1e8700 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.922819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf852edd-357c-4fc9-a180-9a3e9c1e8700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.923673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c30bd-1f47-451f-8a9d-07537d462fc5 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:19.957023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc4bd6-16ef-47db-a938-d486bc843046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:19.957679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc4bd6-16ef-47db-a938-d486bc843046 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:19.957962 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc4bd6-16ef-47db-a938-d486bc843046 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:19.958239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc4bd6-16ef-47db-a938-d486bc843046 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:19.958960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc4bd6-16ef-47db-a938-d486bc843046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:19.959266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc4bd6-16ef-47db-a938-d486bc843046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.961197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc4bd6-16ef-47db-a938-d486bc843046 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:19.972432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 245/511] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:19.973061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc4bd6-16ef-47db-a938-d486bc843046 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:19.993135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf852edd-357c-4fc9-a180-9a3e9c1e8700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:19.994629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf852edd-357c-4fc9-a180-9a3e9c1e8700 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.070786 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc4bd6-16ef-47db-a938-d486bc843046 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.088171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:20.092436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 267/512] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:20.093315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf852edd-357c-4fc9-a180-9a3e9c1e8700 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.108686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:20.112427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58d508-5cba-4ec9-a573-94dcdf6f24a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.113130 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d508-5cba-4ec9-a573-94dcdf6f24a3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.113410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d508-5cba-4ec9-a573-94dcdf6f24a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.113730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d508-5cba-4ec9-a573-94dcdf6f24a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.114395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58d508-5cba-4ec9-a573-94dcdf6f24a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:20.114696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58d508-5cba-4ec9-a573-94dcdf6f24a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.116079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58d508-5cba-4ec9-a573-94dcdf6f24a3 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.130363 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 246/513] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:20.131247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58d508-5cba-4ec9-a573-94dcdf6f24a3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.196298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf852edd-357c-4fc9-a180-9a3e9c1e8700 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.229676 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cfb58-6b34-4406-acb0-a7dbd7416fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.230338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cfb58-6b34-4406-acb0-a7dbd7416fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.230619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cfb58-6b34-4406-acb0-a7dbd7416fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.230893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cfb58-6b34-4406-acb0-a7dbd7416fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.231606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cfb58-6b34-4406-acb0-a7dbd7416fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:20.231706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58d508-5cba-4ec9-a573-94dcdf6f24a3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.231906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cfb58-6b34-4406-acb0-a7dbd7416fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.233294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cfb58-6b34-4406-acb0-a7dbd7416fe9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.244195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 268/514] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:20.245017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cfb58-6b34-4406-acb0-a7dbd7416fe9 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.270018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6d25a-cca4-4107-9700-9ca5b53816eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.270830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6d25a-cca4-4107-9700-9ca5b53816eb None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.271114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6d25a-cca4-4107-9700-9ca5b53816eb None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.271409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6d25a-cca4-4107-9700-9ca5b53816eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.284226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6d25a-cca4-4107-9700-9ca5b53816eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.285761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6d25a-cca4-4107-9700-9ca5b53816eb None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-26f6d25a-cca4-4107-9700-9ca5b53816eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:20.303291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:20.304143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:20.305049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:20.305049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:20.305049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:20.305049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:20.305256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 247/515] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:20.306038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6d25a-cca4-4107-9700-9ca5b53816eb None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.326954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:20.336649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:20.353498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cfb58-6b34-4406-acb0-a7dbd7416fe9 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.386772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fa25a-8749-405a-9e37-1c29f3b5698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.387667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa25a-8749-405a-9e37-1c29f3b5698e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.387942 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa25a-8749-405a-9e37-1c29f3b5698e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.388210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa25a-8749-405a-9e37-1c29f3b5698e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.395467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fa25a-8749-405a-9e37-1c29f3b5698e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.396919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fa25a-8749-405a-9e37-1c29f3b5698e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.409097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6d25a-cca4-4107-9700-9ca5b53816eb None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-f77fa25a-8749-405a-9e37-1c29f3b5698e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:20.411242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:20.412226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:20.413146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 269/516] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:20.413709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fa25a-8749-405a-9e37-1c29f3b5698e None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.444820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e193073-8aa2-45d0-8d01-e30f7bb66e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.445945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e193073-8aa2-45d0-8d01-e30f7bb66e54 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.446240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e193073-8aa2-45d0-8d01-e30f7bb66e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.446515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e193073-8aa2-45d0-8d01-e30f7bb66e54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.456368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e193073-8aa2-45d0-8d01-e30f7bb66e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.457837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e193073-8aa2-45d0-8d01-e30f7bb66e54 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-8e193073-8aa2-45d0-8d01-e30f7bb66e54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:20.475245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:20.476212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:20.477944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:20.477944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:20.477944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:20.477944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:20.477944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 248/517] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:20.478206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e193073-8aa2-45d0-8d01-e30f7bb66e54 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.519375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fa25a-8749-405a-9e37-1c29f3b5698e None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.552657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98570-a70a-4a23-92be-f8ab1537cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.553345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98570-a70a-4a23-92be-f8ab1537cb70 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.553625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98570-a70a-4a23-92be-f8ab1537cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.553901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98570-a70a-4a23-92be-f8ab1537cb70 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.555246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b98570-a70a-4a23-92be-f8ab1537cb70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.556678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b98570-a70a-4a23-92be-f8ab1537cb70 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.560444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:20.563599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:20.577157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 270/518] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:16:20 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:16:20.577961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98570-a70a-4a23-92be-f8ab1537cb70 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.581223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e193073-8aa2-45d0-8d01-e30f7bb66e54 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.627218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d46dd-2f4d-49cb-83ef-9233e5d4a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.628142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d46dd-2f4d-49cb-83ef-9233e5d4a9fc None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.628486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d46dd-2f4d-49cb-83ef-9233e5d4a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.628832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d46dd-2f4d-49cb-83ef-9233e5d4a9fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.637490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d46dd-2f4d-49cb-83ef-9233e5d4a9fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.638989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d46dd-2f4d-49cb-83ef-9233e5d4a9fc None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-196d46dd-2f4d-49cb-83ef-9233e5d4a9fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:20.658691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:20.659665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:20.660527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 249/519] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:20.661148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d46dd-2f4d-49cb-83ef-9233e5d4a9fc None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.704412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98570-a70a-4a23-92be-f8ab1537cb70 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.743131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4bf61-58f1-40bf-b596-26683b2bba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.743823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4bf61-58f1-40bf-b596-26683b2bba7b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.744106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4bf61-58f1-40bf-b596-26683b2bba7b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.744387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4bf61-58f1-40bf-b596-26683b2bba7b None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.745156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4bf61-58f1-40bf-b596-26683b2bba7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:20.745464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4bf61-58f1-40bf-b596-26683b2bba7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.747106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4bf61-58f1-40bf-b596-26683b2bba7b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.764252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 271/520] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:20.764252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4bf61-58f1-40bf-b596-26683b2bba7b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.783764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d46dd-2f4d-49cb-83ef-9233e5d4a9fc None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.823770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:20.832185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62282-0a6b-4194-b221-8caeeb3613e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.832861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62282-0a6b-4194-b221-8caeeb3613e6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.833166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62282-0a6b-4194-b221-8caeeb3613e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.833500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62282-0a6b-4194-b221-8caeeb3613e6 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.834365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d62282-0a6b-4194-b221-8caeeb3613e6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:20.834671 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d62282-0a6b-4194-b221-8caeeb3613e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.836709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d62282-0a6b-4194-b221-8caeeb3613e6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.872378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:20.876931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 250/521] 10.222.0.30 () {56 vars in 1139 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:20.877861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62282-0a6b-4194-b221-8caeeb3613e6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.889539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4bf61-58f1-40bf-b596-26683b2bba7b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:20.928115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bcf08-a813-4cd7-a5f7-022d9313246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:20.929422 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bcf08-a813-4cd7-a5f7-022d9313246d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:20.929894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bcf08-a813-4cd7-a5f7-022d9313246d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:20.930361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bcf08-a813-4cd7-a5f7-022d9313246d None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:20.931260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bcf08-a813-4cd7-a5f7-022d9313246d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:20.931886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bcf08-a813-4cd7-a5f7-022d9313246d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:20.933321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bcf08-a813-4cd7-a5f7-022d9313246d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:20.946494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 272/522] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:20.947319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bcf08-a813-4cd7-a5f7-022d9313246d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:20.984274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62282-0a6b-4194-b221-8caeeb3613e6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:21.028605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dc609-19d3-4f43-b16d-77df2697253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:21.029652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dc609-19d3-4f43-b16d-77df2697253c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:21.030135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dc609-19d3-4f43-b16d-77df2697253c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:21.030621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dc609-19d3-4f43-b16d-77df2697253c None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:21.031678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dc609-19d3-4f43-b16d-77df2697253c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:21.032692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dc609-19d3-4f43-b16d-77df2697253c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:21.034848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dc609-19d3-4f43-b16d-77df2697253c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:21.057332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bcf08-a813-4cd7-a5f7-022d9313246d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:21.061039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 251/523] 10.222.0.30 () {56 vars in 1137 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:21.062002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dc609-19d3-4f43-b16d-77df2697253c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:21.073132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:21.099502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e435c53-d60a-41de-92b0-c326c7f0e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:21.100303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e435c53-d60a-41de-92b0-c326c7f0e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:21.100581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e435c53-d60a-41de-92b0-c326c7f0e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:21.100853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e435c53-d60a-41de-92b0-c326c7f0e0f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:21.108469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e435c53-d60a-41de-92b0-c326c7f0e0f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:21.109947 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e435c53-d60a-41de-92b0-c326c7f0e0f6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:21.123807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-7e435c53-d60a-41de-92b0-c326c7f0e0f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:21.126384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:21.127183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:21.128069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:21.128069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:21.128069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:21.128069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:21.128069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:21.128069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:21.128069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:21.128069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:21.128069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 273/524] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:21.128514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e435c53-d60a-41de-92b0-c326c7f0e0f6 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:21.177466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dc609-19d3-4f43-b16d-77df2697253c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:21.214507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86158d9c-711f-4838-a03e-beddf57413ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:21.215160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86158d9c-711f-4838-a03e-beddf57413ed None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:21.215443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86158d9c-711f-4838-a03e-beddf57413ed None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:21.215744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86158d9c-711f-4838-a03e-beddf57413ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:21.231643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e435c53-d60a-41de-92b0-c326c7f0e0f6 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:21.268292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce35e10-a5a0-4cf2-95c5-71ce45ba3e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:21.269132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce35e10-a5a0-4cf2-95c5-71ce45ba3e64 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:21.269416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce35e10-a5a0-4cf2-95c5-71ce45ba3e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:21.269686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce35e10-a5a0-4cf2-95c5-71ce45ba3e64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:21.277203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce35e10-a5a0-4cf2-95c5-71ce45ba3e64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:21.278646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce35e10-a5a0-4cf2-95c5-71ce45ba3e64 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:21.290415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86158d9c-711f-4838-a03e-beddf57413ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:21.292130 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86158d9c-711f-4838-a03e-beddf57413ed None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-5ce35e10-a5a0-4cf2-95c5-71ce45ba3e64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:21.293228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:21.294095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:21.295204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:21.295204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:21.295204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:21.295204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:21.295204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:21.295204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:21.295204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:21.295204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:21.299029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 274/525] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:21.299933 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce35e10-a5a0-4cf2-95c5-71ce45ba3e64 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:21.333165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:21.358647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:21.406838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 252/526] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:21.407883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86158d9c-711f-4838-a03e-beddf57413ed None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:21.408004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce35e10-a5a0-4cf2-95c5-71ce45ba3e64 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:21.448282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67377c-5334-4eff-8a92-8a0b6a8fad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:21.448987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67377c-5334-4eff-8a92-8a0b6a8fad34 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:21.449276 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67377c-5334-4eff-8a92-8a0b6a8fad34 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:21.449595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67377c-5334-4eff-8a92-8a0b6a8fad34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:21.523214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86158d9c-711f-4838-a03e-beddf57413ed None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:21.552141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67377c-5334-4eff-8a92-8a0b6a8fad34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:21.553623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67377c-5334-4eff-8a92-8a0b6a8fad34 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:21.576574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ce476-5f74-409d-a39f-97dc0490bf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:21.577682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ce476-5f74-409d-a39f-97dc0490bf88 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:21.578178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ce476-5f74-409d-a39f-97dc0490bf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:21.578663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ce476-5f74-409d-a39f-97dc0490bf88 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:21.580169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ce476-5f74-409d-a39f-97dc0490bf88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:21.580702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ce476-5f74-409d-a39f-97dc0490bf88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:21.583444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ce476-5f74-409d-a39f-97dc0490bf88 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:21.584698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:21.614292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 253/527] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:21.620376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ce476-5f74-409d-a39f-97dc0490bf88 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:21.641209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:21.672989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 275/528] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:21.673773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67377c-5334-4eff-8a92-8a0b6a8fad34 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:21.734336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ce476-5f74-409d-a39f-97dc0490bf88 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:21.770174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943cf775-5ed0-4d07-a736-136ed8061c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:21.770843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cf775-5ed0-4d07-a736-136ed8061c84 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:21.771122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cf775-5ed0-4d07-a736-136ed8061c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:21.771396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cf775-5ed0-4d07-a736-136ed8061c84 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:21.772184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943cf775-5ed0-4d07-a736-136ed8061c84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:21.772494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943cf775-5ed0-4d07-a736-136ed8061c84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:21.774019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943cf775-5ed0-4d07-a736-136ed8061c84 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:21.786130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67377c-5334-4eff-8a92-8a0b6a8fad34 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:21.787577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 254/529] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:21.788488 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943cf775-5ed0-4d07-a736-136ed8061c84 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:21.827636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:21.835843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec4614-f7ed-401e-b26b-8daa2778f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:21.837730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec4614-f7ed-401e-b26b-8daa2778f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:21.838019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec4614-f7ed-401e-b26b-8daa2778f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:21.838296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec4614-f7ed-401e-b26b-8daa2778f2e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:21.850759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec4614-f7ed-401e-b26b-8daa2778f2e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:21.852993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec4614-f7ed-401e-b26b-8daa2778f2e5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-30ec4614-f7ed-401e-b26b-8daa2778f2e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:21.871878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:21.872741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:21.873591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:21.873591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:21.873591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:21.873591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:21.873591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 276/530] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:21.874353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec4614-f7ed-401e-b26b-8daa2778f2e5 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:21.885287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:21.909367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943cf775-5ed0-4d07-a736-136ed8061c84 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:21.947055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153cc77-37f9-4173-ad5a-6d1f47f0d7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:21.947639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153cc77-37f9-4173-ad5a-6d1f47f0d7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:21.947922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153cc77-37f9-4173-ad5a-6d1f47f0d7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:21.948200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153cc77-37f9-4173-ad5a-6d1f47f0d7f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:21.949065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153cc77-37f9-4173-ad5a-6d1f47f0d7f7 None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=fb0e53d4b3774959820ac72824b3d4c2) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:21.949376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153cc77-37f9-4173-ad5a-6d1f47f0d7f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:21.950845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153cc77-37f9-4173-ad5a-6d1f47f0d7f7 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:21.964902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 255/531] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:21 2022] GET /identity/v3/role_assignments?user.id=fb0e53d4b3774959820ac72824b3d4c2&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:21.964902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153cc77-37f9-4173-ad5a-6d1f47f0d7f7 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:22.000618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec4614-f7ed-401e-b26b-8daa2778f2e5 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:22.039920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be293fc-a625-4d53-b659-78b3970230b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:22.040731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be293fc-a625-4d53-b659-78b3970230b7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:22.041090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be293fc-a625-4d53-b659-78b3970230b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:22.041367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be293fc-a625-4d53-b659-78b3970230b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:22.056282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be293fc-a625-4d53-b659-78b3970230b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:22.057808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be293fc-a625-4d53-b659-78b3970230b7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-7be293fc-a625-4d53-b659-78b3970230b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:22.087762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:22.088661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:22.089533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:22.089533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:22.089533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:22.089533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:22.090628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 277/532] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:22.092322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be293fc-a625-4d53-b659-78b3970230b7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:22.099044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153cc77-37f9-4173-ad5a-6d1f47f0d7f7 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:22.142276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b97dcf-73ce-44c8-ac8c-e254e9e5da4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:22.142955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b97dcf-73ce-44c8-ac8c-e254e9e5da4b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:22.143237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b97dcf-73ce-44c8-ac8c-e254e9e5da4b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:22.145665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b97dcf-73ce-44c8-ac8c-e254e9e5da4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:22.145665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b97dcf-73ce-44c8-ac8c-e254e9e5da4b None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=232cfaefaf584ac59720713e02cf2c2d) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:22.145665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b97dcf-73ce-44c8-ac8c-e254e9e5da4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:22.147398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b97dcf-73ce-44c8-ac8c-e254e9e5da4b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:22.152794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:22.155639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 256/533] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:21 2022] GET /identity/v3/role_assignments?user.id=232cfaefaf584ac59720713e02cf2c2d&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:22.156663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b97dcf-73ce-44c8-ac8c-e254e9e5da4b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:22.167518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:22.237154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be293fc-a625-4d53-b659-78b3970230b7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:22.279609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bfe5b-fcfc-4eb9-8114-61f55e7ab366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:22.280673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bfe5b-fcfc-4eb9-8114-61f55e7ab366 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:22.281195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bfe5b-fcfc-4eb9-8114-61f55e7ab366 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:22.281699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bfe5b-fcfc-4eb9-8114-61f55e7ab366 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:22.282298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b97dcf-73ce-44c8-ac8c-e254e9e5da4b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:22.282869 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94bfe5b-fcfc-4eb9-8114-61f55e7ab366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:22.283387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94bfe5b-fcfc-4eb9-8114-61f55e7ab366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:22.285260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94bfe5b-fcfc-4eb9-8114-61f55e7ab366 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:22.299172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 278/534] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:22.306538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bfe5b-fcfc-4eb9-8114-61f55e7ab366 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:22.374813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ff34d-75c4-4666-a1c0-c2d2e8cc2b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:22.374813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ff34d-75c4-4666-a1c0-c2d2e8cc2b6b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:22.375426 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ff34d-75c4-4666-a1c0-c2d2e8cc2b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:22.375426 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ff34d-75c4-4666-a1c0-c2d2e8cc2b6b None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:22.376329 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ff34d-75c4-4666-a1c0-c2d2e8cc2b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:22.376436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ff34d-75c4-4666-a1c0-c2d2e8cc2b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:22.377875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ff34d-75c4-4666-a1c0-c2d2e8cc2b6b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:22.396163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 257/535] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:22.405651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ff34d-75c4-4666-a1c0-c2d2e8cc2b6b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:22.441554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:22.448956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bfe5b-fcfc-4eb9-8114-61f55e7ab366 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:22.484819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa4424-d5ae-4dc1-8780-87b1f6545341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:22.485645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa4424-d5ae-4dc1-8780-87b1f6545341 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:22.485925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa4424-d5ae-4dc1-8780-87b1f6545341 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:22.486200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa4424-d5ae-4dc1-8780-87b1f6545341 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:22.493953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa4424-d5ae-4dc1-8780-87b1f6545341 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:22.495410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa4424-d5ae-4dc1-8780-87b1f6545341 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-e6aa4424-d5ae-4dc1-8780-87b1f6545341 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:22.511832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:22.512778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:22.513719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:22.513719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:22.513719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:22.513719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:22.513719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:22.513719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:22.513719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:22.513719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:22.518219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 279/536] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:22.521418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa4424-d5ae-4dc1-8780-87b1f6545341 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:22.567688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:22.626623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ff34d-75c4-4666-a1c0-c2d2e8cc2b6b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:22.675076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3877d-d699-4f9f-9b05-bf5573281f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:22.675884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3877d-d699-4f9f-9b05-bf5573281f3f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:22.676169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3877d-d699-4f9f-9b05-bf5573281f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:22.676445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3877d-d699-4f9f-9b05-bf5573281f3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:22.692384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3877d-d699-4f9f-9b05-bf5573281f3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:22.695004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3877d-d699-4f9f-9b05-bf5573281f3f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:22.697164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa4424-d5ae-4dc1-8780-87b1f6545341 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-94b3877d-d699-4f9f-9b05-bf5573281f3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:22.721956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:22.722841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:22.723657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:22.723657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:22.723657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:22.723657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:22.723657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:22.723657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:22.723657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:22.723657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:22.730902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 258/537] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:22.738087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3877d-d699-4f9f-9b05-bf5573281f3f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:22.768738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff658c-408e-458a-9e3c-b42a8a1fa313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:22.769907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff658c-408e-458a-9e3c-b42a8a1fa313 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:22.770246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff658c-408e-458a-9e3c-b42a8a1fa313 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:22.770530 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff658c-408e-458a-9e3c-b42a8a1fa313 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:22.771353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff658c-408e-458a-9e3c-b42a8a1fa313 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:22.771725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff658c-408e-458a-9e3c-b42a8a1fa313 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:22.775782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff658c-408e-458a-9e3c-b42a8a1fa313 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:22.805322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 280/538] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:22.808418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff658c-408e-458a-9e3c-b42a8a1fa313 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:22.841343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:22.968452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3877d-d699-4f9f-9b05-bf5573281f3f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:22.999104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff658c-408e-458a-9e3c-b42a8a1fa313 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:23.008312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:23.020936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d82c9e-be06-49f4-9e15-88db30e407a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:23.021871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d82c9e-be06-49f4-9e15-88db30e407a5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:23.022152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d82c9e-be06-49f4-9e15-88db30e407a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:23.022428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d82c9e-be06-49f4-9e15-88db30e407a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:23.024362 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d82c9e-be06-49f4-9e15-88db30e407a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:23.024948 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d82c9e-be06-49f4-9e15-88db30e407a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:23.026705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d82c9e-be06-49f4-9e15-88db30e407a5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:23.034592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5a18b-4e1e-4c03-ab92-e78147118a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:23.035350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5a18b-4e1e-4c03-ab92-e78147118a45 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:23.035639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5a18b-4e1e-4c03-ab92-e78147118a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:23.035916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5a18b-4e1e-4c03-ab92-e78147118a45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:23.036751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5a18b-4e1e-4c03-ab92-e78147118a45 None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=68d85e20dc884cd0bad5bb6c304776ca) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:23.037109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5a18b-4e1e-4c03-ab92-e78147118a45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:23.038682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5a18b-4e1e-4c03-ab92-e78147118a45 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:23.042346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 259/539] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:23.046392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bab65-7da6-4514-8cbe-694cd2537756 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:23.046679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bab65-7da6-4514-8cbe-694cd2537756 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:23.046954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bab65-7da6-4514-8cbe-694cd2537756 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:23.048253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 260/540] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:23.049197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bab65-7da6-4514-8cbe-694cd2537756 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:23.051280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 281/541] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:22 2022] GET /identity/v3/role_assignments?user.id=68d85e20dc884cd0bad5bb6c304776ca&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:23.053795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6074d9b-a84c-4da9-9376-4d1034cf43e3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:23.054084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6074d9b-a84c-4da9-9376-4d1034cf43e3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:23.054365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6074d9b-a84c-4da9-9376-4d1034cf43e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:23.074398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d6074d9b-a84c-4da9-9376-4d1034cf43e3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:23.075625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d6074d9b-a84c-4da9-9376-4d1034cf43e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:23.156531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 282/542] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:23.182354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6074d9b-a84c-4da9-9376-4d1034cf43e3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:23.194216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:23.284945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bab65-7da6-4514-8cbe-694cd2537756 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:23.298013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6074d9b-a84c-4da9-9376-4d1034cf43e3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:23.322183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a8f6a-32d2-467d-89f4-dfa772df928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:23.322861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a8f6a-32d2-467d-89f4-dfa772df928c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:23.323125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a8f6a-32d2-467d-89f4-dfa772df928c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:23.323503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a8f6a-32d2-467d-89f4-dfa772df928c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:23.324443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a8f6a-32d2-467d-89f4-dfa772df928c None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=36daf80a4cbc4cfc8962c7924433d257) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:23.324759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a8f6a-32d2-467d-89f4-dfa772df928c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:23.326236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a8f6a-32d2-467d-89f4-dfa772df928c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:23.335568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc340020-779c-44bc-afd6-38244eb3091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:23.336737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc340020-779c-44bc-afd6-38244eb3091a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:23.337164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc340020-779c-44bc-afd6-38244eb3091a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:23.337440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc340020-779c-44bc-afd6-38244eb3091a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:23.339685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 261/543] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:23 2022] GET /identity/v3/role_assignments?user.id=36daf80a4cbc4cfc8962c7924433d257&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:23.358958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc340020-779c-44bc-afd6-38244eb3091a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-cc340020-779c-44bc-afd6-38244eb3091a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:23.366907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:23.368455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:23.369255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:23.374256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 283/544] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:23.381014 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a8f6a-32d2-467d-89f4-dfa772df928c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:23.467069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:23.528162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a8f6a-32d2-467d-89f4-dfa772df928c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:23.585852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635e96e-95c4-4754-86e2-7e68c09283ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:23.586654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635e96e-95c4-4754-86e2-7e68c09283ef None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:23.587055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635e96e-95c4-4754-86e2-7e68c09283ef None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:23.587220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635e96e-95c4-4754-86e2-7e68c09283ef None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:23.588005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635e96e-95c4-4754-86e2-7e68c09283ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:23.588316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635e96e-95c4-4754-86e2-7e68c09283ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:23.589807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635e96e-95c4-4754-86e2-7e68c09283ef None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:23.603432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 262/545] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:23.611676 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc340020-779c-44bc-afd6-38244eb3091a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:23.687519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:23.725070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc340020-779c-44bc-afd6-38244eb3091a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:23.760369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85fade-6aec-496e-92a8-262542064806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:23.761196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85fade-6aec-496e-92a8-262542064806 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:23.761480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85fade-6aec-496e-92a8-262542064806 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:23.761752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85fade-6aec-496e-92a8-262542064806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:23.772302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85fade-6aec-496e-92a8-262542064806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:23.774555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85fade-6aec-496e-92a8-262542064806 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-cf85fade-6aec-496e-92a8-262542064806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:23.792100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:23.792965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:23.793782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:23.793782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:23.793782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:23.793782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:23.793782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 284/546] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:23.802042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635e96e-95c4-4754-86e2-7e68c09283ef None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:23.924079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635e96e-95c4-4754-86e2-7e68c09283ef None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:23.938127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:23.960546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089d72c-143a-40bb-a9eb-53ec801fc97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:23.961868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d72c-143a-40bb-a9eb-53ec801fc97f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:23.962520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d72c-143a-40bb-a9eb-53ec801fc97f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:23.963060 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d72c-143a-40bb-a9eb-53ec801fc97f None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:23.964214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089d72c-143a-40bb-a9eb-53ec801fc97f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:23.964554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089d72c-143a-40bb-a9eb-53ec801fc97f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:23.966019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089d72c-143a-40bb-a9eb-53ec801fc97f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:23.979112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 263/547] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:23.984174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85fade-6aec-496e-92a8-262542064806 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:24.153012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85fade-6aec-496e-92a8-262542064806 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:24.181082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:24.197955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513aa95-7436-4361-9c98-88808b24e61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:24.198761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513aa95-7436-4361-9c98-88808b24e61a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:24.199042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513aa95-7436-4361-9c98-88808b24e61a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:24.199317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513aa95-7436-4361-9c98-88808b24e61a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:24.213517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513aa95-7436-4361-9c98-88808b24e61a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:24.214958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513aa95-7436-4361-9c98-88808b24e61a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-f513aa95-7436-4361-9c98-88808b24e61a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:24.235636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:24.237105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:24.238144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:24.244908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 285/548] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:24.246058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089d72c-143a-40bb-a9eb-53ec801fc97f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:24.377309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089d72c-143a-40bb-a9eb-53ec801fc97f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:24.413149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271ad5d-9499-451d-8aca-a83e1795adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:24.414009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271ad5d-9499-451d-8aca-a83e1795adfb None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:24.414463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271ad5d-9499-451d-8aca-a83e1795adfb None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:24.414832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271ad5d-9499-451d-8aca-a83e1795adfb None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:24.415704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271ad5d-9499-451d-8aca-a83e1795adfb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:24.416017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271ad5d-9499-451d-8aca-a83e1795adfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:24.417560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271ad5d-9499-451d-8aca-a83e1795adfb None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:24.440698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 264/549] 10.222.0.30 () {56 vars in 1135 bytes} [Mon May 23 15:16:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:24.446233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513aa95-7436-4361-9c98-88808b24e61a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:24.556175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513aa95-7436-4361-9c98-88808b24e61a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:24.597632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feaf354-43d2-44d6-a41c-f72e51c6524d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:24.598298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feaf354-43d2-44d6-a41c-f72e51c6524d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:24.598585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feaf354-43d2-44d6-a41c-f72e51c6524d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:24.598864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feaf354-43d2-44d6-a41c-f72e51c6524d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:24.620018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae997429-f6f0-4838-8490-e2b4ccc51d53 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:24.620609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae997429-f6f0-4838-8490-e2b4ccc51d53 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:24.621139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae997429-f6f0-4838-8490-e2b4ccc51d53 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:24.623210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 265/550] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:24.628630 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624240a-75af-48a3-b26c-7e489cb0dbfc None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:24.629263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624240a-75af-48a3-b26c-7e489cb0dbfc None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:24.629749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624240a-75af-48a3-b26c-7e489cb0dbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:24.648342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4624240a-75af-48a3-b26c-7e489cb0dbfc None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:24.649994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4624240a-75af-48a3-b26c-7e489cb0dbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:24.659701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:24.678413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:24.696977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feaf354-43d2-44d6-a41c-f72e51c6524d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:24.696977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feaf354-43d2-44d6-a41c-f72e51c6524d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:24.806351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 286/551] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:24.814874 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feaf354-43d2-44d6-a41c-f72e51c6524d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:24.819227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 266/552] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:24.867188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624240a-75af-48a3-b26c-7e489cb0dbfc None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:24.912045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:24.935306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feaf354-43d2-44d6-a41c-f72e51c6524d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:24.973393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02888577-2c1b-4ab5-b170-9da2bd7fbf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:24.974595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02888577-2c1b-4ab5-b170-9da2bd7fbf14 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:24.975102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02888577-2c1b-4ab5-b170-9da2bd7fbf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:24.975594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02888577-2c1b-4ab5-b170-9da2bd7fbf14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:24.987231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02888577-2c1b-4ab5-b170-9da2bd7fbf14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:24.988274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624240a-75af-48a3-b26c-7e489cb0dbfc None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:24.989288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02888577-2c1b-4ab5-b170-9da2bd7fbf14 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-02888577-2c1b-4ab5-b170-9da2bd7fbf14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:25.011645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:25.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:25.013631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:25.013631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:25.013631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:25.013631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:25.015601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 287/553] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:25.017050 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b876f22-30ee-4c86-88b2-96f590df4a38 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.017482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b876f22-30ee-4c86-88b2-96f590df4a38 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.017921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b876f22-30ee-4c86-88b2-96f590df4a38 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.019588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 288/554] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:25.020918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b876f22-30ee-4c86-88b2-96f590df4a38 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:25.036736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff72ac-0a4a-47a3-9c3b-4f3b5a620476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:25.037796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff72ac-0a4a-47a3-9c3b-4f3b5a620476 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.038518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff72ac-0a4a-47a3-9c3b-4f3b5a620476 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.039389 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff72ac-0a4a-47a3-9c3b-4f3b5a620476 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.052545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff72ac-0a4a-47a3-9c3b-4f3b5a620476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-33ff72ac-0a4a-47a3-9c3b-4f3b5a620476 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:25.063276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:25.064184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:25.065293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 267/555] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:25.066727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad36d6a-eafb-4c5c-a31f-9d84fb05281b None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.067078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad36d6a-eafb-4c5c-a31f-9d84fb05281b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.067419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad36d6a-eafb-4c5c-a31f-9d84fb05281b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.098740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-7ad36d6a-eafb-4c5c-a31f-9d84fb05281b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:25.100680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7ad36d6a-eafb-4c5c-a31f-9d84fb05281b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:25.122456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:25.152385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b876f22-30ee-4c86-88b2-96f590df4a38 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:25.192747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:25.200275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807beeb-3af7-4bc1-b8b9-2c076e94e06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:25.201300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807beeb-3af7-4bc1-b8b9-2c076e94e06c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.201763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807beeb-3af7-4bc1-b8b9-2c076e94e06c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.202215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807beeb-3af7-4bc1-b8b9-2c076e94e06c None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.203115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807beeb-3af7-4bc1-b8b9-2c076e94e06c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:25.203621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807beeb-3af7-4bc1-b8b9-2c076e94e06c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:25.205294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807beeb-3af7-4bc1-b8b9-2c076e94e06c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:25.207421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 268/556] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:25.209091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad36d6a-eafb-4c5c-a31f-9d84fb05281b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:25.223333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 289/557] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:25.225008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807beeb-3af7-4bc1-b8b9-2c076e94e06c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:25.335464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad36d6a-eafb-4c5c-a31f-9d84fb05281b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:25.349277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807beeb-3af7-4bc1-b8b9-2c076e94e06c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:25.372167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fcd6b-4ea6-415f-bfb2-616c514ea8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:25.372962 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fcd6b-4ea6-415f-bfb2-616c514ea8bb None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.373334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fcd6b-4ea6-415f-bfb2-616c514ea8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.373679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fcd6b-4ea6-415f-bfb2-616c514ea8bb None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.374490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fcd6b-4ea6-415f-bfb2-616c514ea8bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:25.374860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fcd6b-4ea6-415f-bfb2-616c514ea8bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:25.376301 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fcd6b-4ea6-415f-bfb2-616c514ea8bb None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:25.386093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 269/558] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:25.387116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fcd6b-4ea6-415f-bfb2-616c514ea8bb None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:25.398980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3766d-f770-444a-a351-f84b99e7205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:25.399789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3766d-f770-444a-a351-f84b99e7205b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.400073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3766d-f770-444a-a351-f84b99e7205b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.400352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3766d-f770-444a-a351-f84b99e7205b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.408263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:25.411701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3766d-f770-444a-a351-f84b99e7205b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:25.413722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3766d-f770-444a-a351-f84b99e7205b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-e6f3766d-f770-444a-a351-f84b99e7205b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:25.435730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:25.437143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 290/559] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:25.438607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3766d-f770-444a-a351-f84b99e7205b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:25.506615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fcd6b-4ea6-415f-bfb2-616c514ea8bb None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:25.516840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:25.547160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12822d-e3df-41e6-866e-7040b2c825c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:25.548080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12822d-e3df-41e6-866e-7040b2c825c5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.548433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12822d-e3df-41e6-866e-7040b2c825c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.548771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12822d-e3df-41e6-866e-7040b2c825c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.560666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12822d-e3df-41e6-866e-7040b2c825c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-1c12822d-e3df-41e6-866e-7040b2c825c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:25.568526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:25.569748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:25.571315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:25.572623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 270/560] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:25.577682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12822d-e3df-41e6-866e-7040b2c825c5 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:25.582283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3766d-f770-444a-a351-f84b99e7205b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:25.629108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc80cd-deb5-4d71-b3d9-ab8a9cdad2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:25.629924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc80cd-deb5-4d71-b3d9-ab8a9cdad2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.630209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc80cd-deb5-4d71-b3d9-ab8a9cdad2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.630486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc80cd-deb5-4d71-b3d9-ab8a9cdad2b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.642977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc80cd-deb5-4d71-b3d9-ab8a9cdad2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:25.644941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc80cd-deb5-4d71-b3d9-ab8a9cdad2b5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-f4bc80cd-deb5-4d71-b3d9-ab8a9cdad2b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:25.669685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:25.671363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:25.672377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:25.672377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:25.672377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:25.672377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:25.674368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:25.680290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 291/561] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:25.682302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc80cd-deb5-4d71-b3d9-ab8a9cdad2b5 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:25.710486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12822d-e3df-41e6-866e-7040b2c825c5 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:25.749875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c1dfd-61b9-464e-b400-40116f222e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:25.750941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c1dfd-61b9-464e-b400-40116f222e46 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.751524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c1dfd-61b9-464e-b400-40116f222e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.752011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c1dfd-61b9-464e-b400-40116f222e46 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.753096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c1dfd-61b9-464e-b400-40116f222e46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:25.753653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c1dfd-61b9-464e-b400-40116f222e46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:25.755555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c1dfd-61b9-464e-b400-40116f222e46 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:25.772201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 271/562] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:25.774294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c1dfd-61b9-464e-b400-40116f222e46 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:25.812927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc80cd-deb5-4d71-b3d9-ab8a9cdad2b5 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:25.816234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:25.857311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561c9b2-db39-4374-9dda-bffcc60bee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:25.857991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561c9b2-db39-4374-9dda-bffcc60bee1a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.858269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561c9b2-db39-4374-9dda-bffcc60bee1a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.858537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561c9b2-db39-4374-9dda-bffcc60bee1a None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.859281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561c9b2-db39-4374-9dda-bffcc60bee1a None admin] RBAC: Adding query filter params (name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:25.859581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561c9b2-db39-4374-9dda-bffcc60bee1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:25.860984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561c9b2-db39-4374-9dda-bffcc60bee1a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:25.875222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 292/563] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:25.876216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561c9b2-db39-4374-9dda-bffcc60bee1a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:25.897935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c1dfd-61b9-464e-b400-40116f222e46 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:25.934595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:25.941758 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb11cc-e4b1-4dc5-8b1a-3705bbe91cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:25.942443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb11cc-e4b1-4dc5-8b1a-3705bbe91cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:25.942724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb11cc-e4b1-4dc5-8b1a-3705bbe91cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:25.942997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb11cc-e4b1-4dc5-8b1a-3705bbe91cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:25.943758 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb11cc-e4b1-4dc5-8b1a-3705bbe91cf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:25.944098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb11cc-e4b1-4dc5-8b1a-3705bbe91cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:25.945599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb11cc-e4b1-4dc5-8b1a-3705bbe91cf2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:25.961457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 272/564] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:25.962992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb11cc-e4b1-4dc5-8b1a-3705bbe91cf2 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.002280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561c9b2-db39-4374-9dda-bffcc60bee1a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.042246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69511b6a-5fc2-4cdf-8e10-e2163279eae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.042823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69511b6a-5fc2-4cdf-8e10-e2163279eae9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.043101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69511b6a-5fc2-4cdf-8e10-e2163279eae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.043371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69511b6a-5fc2-4cdf-8e10-e2163279eae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.044192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69511b6a-5fc2-4cdf-8e10-e2163279eae9 None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=ff6c489d950a4370bbf01866d2fc8fc3) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:26.044495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69511b6a-5fc2-4cdf-8e10-e2163279eae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:26.045950 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69511b6a-5fc2-4cdf-8e10-e2163279eae9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:26.051597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 293/565] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/role_assignments?user.id=ff6c489d950a4370bbf01866d2fc8fc3&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:26.052809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714c94-e529-4ccd-8d38-c48e981677fd None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.053116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714c94-e529-4ccd-8d38-c48e981677fd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.053390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714c94-e529-4ccd-8d38-c48e981677fd None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.057017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 294/566] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:26.057017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714c94-e529-4ccd-8d38-c48e981677fd None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.082435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb11cc-e4b1-4dc5-8b1a-3705bbe91cf2 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.093755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:26.131822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b962fe-1a39-4bbc-91b1-4e4c20e5346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.133511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b962fe-1a39-4bbc-91b1-4e4c20e5346d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.133880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b962fe-1a39-4bbc-91b1-4e4c20e5346d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.134156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b962fe-1a39-4bbc-91b1-4e4c20e5346d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.150137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b962fe-1a39-4bbc-91b1-4e4c20e5346d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:26.151664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b962fe-1a39-4bbc-91b1-4e4c20e5346d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-58b962fe-1a39-4bbc-91b1-4e4c20e5346d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:26.178134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:26.179148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:26.180784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:26.180784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:26.180784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:26.180784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:26.180784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 273/567] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:26.181674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee261-bd38-4f1b-a400-5289b02a7dd6 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.181674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee261-bd38-4f1b-a400-5289b02a7dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.181925 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee261-bd38-4f1b-a400-5289b02a7dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.186387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714c94-e529-4ccd-8d38-c48e981677fd None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.202562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-3c5ee261-bd38-4f1b-a400-5289b02a7dd6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:26.204087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3c5ee261-bd38-4f1b-a400-5289b02a7dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:26.225663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e14c9c-a0eb-42e6-b346-47770b1f6879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.227139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e14c9c-a0eb-42e6-b346-47770b1f6879 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.227493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e14c9c-a0eb-42e6-b346-47770b1f6879 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.227788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e14c9c-a0eb-42e6-b346-47770b1f6879 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.236416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e14c9c-a0eb-42e6-b346-47770b1f6879 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:26.237920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e14c9c-a0eb-42e6-b346-47770b1f6879 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:26.252792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-d8e14c9c-a0eb-42e6-b346-47770b1f6879 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:26.254008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:26.254946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:26.255877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 295/568] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:26.257495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38def02b-a82d-4911-9d8c-15eb0f1c0b7c None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.257495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38def02b-a82d-4911-9d8c-15eb0f1c0b7c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.257495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38def02b-a82d-4911-9d8c-15eb0f1c0b7c None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.259298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 296/569] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:26.259587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38def02b-a82d-4911-9d8c-15eb0f1c0b7c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.304972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 274/570] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:26.304972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee261-bd38-4f1b-a400-5289b02a7dd6 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.370369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:26.383238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38def02b-a82d-4911-9d8c-15eb0f1c0b7c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.413623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee261-bd38-4f1b-a400-5289b02a7dd6 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.419843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d457d-ff19-4c41-a627-0898e2e13632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.420509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d457d-ff19-4c41-a627-0898e2e13632 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.420789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d457d-ff19-4c41-a627-0898e2e13632 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.421105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d457d-ff19-4c41-a627-0898e2e13632 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.422227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d457d-ff19-4c41-a627-0898e2e13632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:26.422531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d457d-ff19-4c41-a627-0898e2e13632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:26.424240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d457d-ff19-4c41-a627-0898e2e13632 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:26.437942 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 297/571] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:26.439320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb249f98-b413-4e44-8aea-6fb19767876c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.439610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb249f98-b413-4e44-8aea-6fb19767876c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.439887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb249f98-b413-4e44-8aea-6fb19767876c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.455466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5114a9a-a9d5-4798-a68f-39ea24ea5892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.456140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5114a9a-a9d5-4798-a68f-39ea24ea5892 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.456430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5114a9a-a9d5-4798-a68f-39ea24ea5892 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.456684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5114a9a-a9d5-4798-a68f-39ea24ea5892 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.457631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5114a9a-a9d5-4798-a68f-39ea24ea5892 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:26.457935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5114a9a-a9d5-4798-a68f-39ea24ea5892 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:26.458445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-fb249f98-b413-4e44-8aea-6fb19767876c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:26.459374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5114a9a-a9d5-4798-a68f-39ea24ea5892 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:26.459622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fb249f98-b413-4e44-8aea-6fb19767876c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:26.483287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 275/572] 10.222.0.30 () {56 vars in 1137 bytes} [Mon May 23 15:16:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:26.484162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5114a9a-a9d5-4798-a68f-39ea24ea5892 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.517897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:26.544889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 298/573] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:26.544889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb249f98-b413-4e44-8aea-6fb19767876c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.593199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5114a9a-a9d5-4798-a68f-39ea24ea5892 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.631187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79a0b5-cb79-4091-bb46-138c666b39cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.631988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79a0b5-cb79-4091-bb46-138c666b39cc None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.632515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79a0b5-cb79-4091-bb46-138c666b39cc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.632515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79a0b5-cb79-4091-bb46-138c666b39cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.637192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:26.644153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79a0b5-cb79-4091-bb46-138c666b39cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-4c79a0b5-cb79-4091-bb46-138c666b39cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:26.651958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:26.653268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 276/574] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:26.654106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79a0b5-cb79-4091-bb46-138c666b39cc None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.663329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb249f98-b413-4e44-8aea-6fb19767876c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.698932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a347844-ff3f-4589-8511-189d2c194533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.699722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a347844-ff3f-4589-8511-189d2c194533 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.700002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a347844-ff3f-4589-8511-189d2c194533 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.700272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a347844-ff3f-4589-8511-189d2c194533 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.707352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a347844-ff3f-4589-8511-189d2c194533 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:26.708768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a347844-ff3f-4589-8511-189d2c194533 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-9a347844-ff3f-4589-8511-189d2c194533 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:26.724041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:26.725058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:26.726025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:26.730017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 299/575] 10.222.0.30 () {56 vars in 1093 bytes} [Mon May 23 15:16:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:26.730751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a347844-ff3f-4589-8511-189d2c194533 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.754815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:26.765641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79a0b5-cb79-4091-bb46-138c666b39cc None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.801785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db028fc-200b-4487-9c8e-68c2d8749daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.802581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db028fc-200b-4487-9c8e-68c2d8749daf None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.802869 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db028fc-200b-4487-9c8e-68c2d8749daf None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.803147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db028fc-200b-4487-9c8e-68c2d8749daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.815015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db028fc-200b-4487-9c8e-68c2d8749daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:26.816484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db028fc-200b-4487-9c8e-68c2d8749daf None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-8db028fc-200b-4487-9c8e-68c2d8749daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:26.834574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:26.835487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:26.836587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:26.836587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:26.836587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:26.836587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:26.836829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a347844-ff3f-4589-8511-189d2c194533 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.840522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 277/576] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:26.841430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db028fc-200b-4487-9c8e-68c2d8749daf None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.873113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56d151-4fb2-4f94-a433-e36af0db7e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.873834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56d151-4fb2-4f94-a433-e36af0db7e9a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.874122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56d151-4fb2-4f94-a433-e36af0db7e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.874390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56d151-4fb2-4f94-a433-e36af0db7e9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.882575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:26.886394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56d151-4fb2-4f94-a433-e36af0db7e9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-fd56d151-4fb2-4f94-a433-e36af0db7e9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:26.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:26.895773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:26.896633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:26.902343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 300/577] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:26.903540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56d151-4fb2-4f94-a433-e36af0db7e9a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:26.952850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db028fc-200b-4487-9c8e-68c2d8749daf None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:26.988778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05265bc6-0b36-458e-b4b6-534d41b48545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:26.989470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265bc6-0b36-458e-b4b6-534d41b48545 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:26.989753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265bc6-0b36-458e-b4b6-534d41b48545 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:26.990030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265bc6-0b36-458e-b4b6-534d41b48545 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:26.990876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05265bc6-0b36-458e-b4b6-534d41b48545 None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:26.991187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05265bc6-0b36-458e-b4b6-534d41b48545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:26.993403 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05265bc6-0b36-458e-b4b6-534d41b48545 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.006941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 278/578] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:27.006941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05265bc6-0b36-458e-b4b6-534d41b48545 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.011338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56d151-4fb2-4f94-a433-e36af0db7e9a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.012585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:27.047770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30379f-395d-435f-a6a4-0ee5953fdfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.048617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30379f-395d-435f-a6a4-0ee5953fdfee None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.048934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30379f-395d-435f-a6a4-0ee5953fdfee None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.049217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30379f-395d-435f-a6a4-0ee5953fdfee None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.050049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30379f-395d-435f-a6a4-0ee5953fdfee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:27.050357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30379f-395d-435f-a6a4-0ee5953fdfee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.051796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30379f-395d-435f-a6a4-0ee5953fdfee None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.069859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 301/579] 10.222.0.30 () {56 vars in 1133 bytes} [Mon May 23 15:16:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:27.075040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30379f-395d-435f-a6a4-0ee5953fdfee None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.112756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05265bc6-0b36-458e-b4b6-534d41b48545 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.139646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:27.151479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70f56f-892d-4b4b-884d-0cffc32ec323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.152142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70f56f-892d-4b4b-884d-0cffc32ec323 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.152425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70f56f-892d-4b4b-884d-0cffc32ec323 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.152700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70f56f-892d-4b4b-884d-0cffc32ec323 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.153469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70f56f-892d-4b4b-884d-0cffc32ec323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:27.153775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70f56f-892d-4b4b-884d-0cffc32ec323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.155151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70f56f-892d-4b4b-884d-0cffc32ec323 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.172536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 279/580] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:27.172536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70f56f-892d-4b4b-884d-0cffc32ec323 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.194870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30379f-395d-435f-a6a4-0ee5953fdfee None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.234392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12757a85-0dfc-41a9-b30c-231aa3bbfe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.235061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12757a85-0dfc-41a9-b30c-231aa3bbfe6f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.235345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12757a85-0dfc-41a9-b30c-231aa3bbfe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.235831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12757a85-0dfc-41a9-b30c-231aa3bbfe6f None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.236370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12757a85-0dfc-41a9-b30c-231aa3bbfe6f None admin] RBAC: Adding query filter params (name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:27.236695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12757a85-0dfc-41a9-b30c-231aa3bbfe6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.238047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12757a85-0dfc-41a9-b30c-231aa3bbfe6f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.239713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:27.257010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 302/581] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:27.257010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12757a85-0dfc-41a9-b30c-231aa3bbfe6f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.275999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70f56f-892d-4b4b-884d-0cffc32ec323 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.310464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2e27a-d819-4726-a1f8-aae7d21441c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.311254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2e27a-d819-4726-a1f8-aae7d21441c5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.311541 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2e27a-d819-4726-a1f8-aae7d21441c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.311822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2e27a-d819-4726-a1f8-aae7d21441c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.322953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2e27a-d819-4726-a1f8-aae7d21441c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.324438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2e27a-d819-4726-a1f8-aae7d21441c5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-ddc2e27a-d819-4726-a1f8-aae7d21441c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:27.343865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:27.344993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:27.345904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:27.345904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:27.345904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:27.345904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:27.349821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 280/582] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:27.350437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2e27a-d819-4726-a1f8-aae7d21441c5 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.372272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12757a85-0dfc-41a9-b30c-231aa3bbfe6f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.393438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:27.408621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f28043-a90b-4120-a3a9-cfd361085776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.409810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f28043-a90b-4120-a3a9-cfd361085776 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.410970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f28043-a90b-4120-a3a9-cfd361085776 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.410970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f28043-a90b-4120-a3a9-cfd361085776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.421876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f28043-a90b-4120-a3a9-cfd361085776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.423314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f28043-a90b-4120-a3a9-cfd361085776 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-81f28043-a90b-4120-a3a9-cfd361085776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:27.440967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:27.441891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:27.442793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:27.442793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:27.442793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:27.442793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:27.442793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 303/583] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:27.443083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f28043-a90b-4120-a3a9-cfd361085776 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.466898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2e27a-d819-4726-a1f8-aae7d21441c5 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.484745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:27.504436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1905d-f7d2-4ba4-a93a-8f61bd492d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.505268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1905d-f7d2-4ba4-a93a-8f61bd492d0c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.505553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1905d-f7d2-4ba4-a93a-8f61bd492d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.505836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1905d-f7d2-4ba4-a93a-8f61bd492d0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.513412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1905d-f7d2-4ba4-a93a-8f61bd492d0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.514877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1905d-f7d2-4ba4-a93a-8f61bd492d0c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-22b1905d-f7d2-4ba4-a93a-8f61bd492d0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:27.529221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:27.530529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 281/584] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:27.531727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1905d-f7d2-4ba4-a93a-8f61bd492d0c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.562843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f28043-a90b-4120-a3a9-cfd361085776 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.604503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb1a4-fea3-4931-92af-3dc239e83a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.605579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb1a4-fea3-4931-92af-3dc239e83a0f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.606009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb1a4-fea3-4931-92af-3dc239e83a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.606492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb1a4-fea3-4931-92af-3dc239e83a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.618948 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58eb1a4-fea3-4931-92af-3dc239e83a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.620759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58eb1a4-fea3-4931-92af-3dc239e83a0f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-e58eb1a4-fea3-4931-92af-3dc239e83a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:27.640427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:27.641411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:27.642329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:27.642329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:27.642329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:27.642329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:27.642329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 304/585] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:27.642645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb1a4-fea3-4931-92af-3dc239e83a0f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.648171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1905d-f7d2-4ba4-a93a-8f61bd492d0c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.673129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:27.685979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a213-3a19-489c-8914-8ab082be1ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.686642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a213-3a19-489c-8914-8ab082be1ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.686930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a213-3a19-489c-8914-8ab082be1ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.687196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a213-3a19-489c-8914-8ab082be1ff8 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.687922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a213-3a19-489c-8914-8ab082be1ff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:27.688229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a213-3a19-489c-8914-8ab082be1ff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.689640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a213-3a19-489c-8914-8ab082be1ff8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.706795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 282/586] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:27.706795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a213-3a19-489c-8914-8ab082be1ff8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.747282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:27.765254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb1a4-fea3-4931-92af-3dc239e83a0f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.801046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd3471-1239-4103-ae92-65378ffd0a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.801717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd3471-1239-4103-ae92-65378ffd0a55 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.802003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd3471-1239-4103-ae92-65378ffd0a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.802276 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd3471-1239-4103-ae92-65378ffd0a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.802998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd3471-1239-4103-ae92-65378ffd0a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:27.803304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd3471-1239-4103-ae92-65378ffd0a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.804784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd3471-1239-4103-ae92-65378ffd0a55 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.812132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a213-3a19-489c-8914-8ab082be1ff8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.822637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 305/587] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:27.822637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd3471-1239-4103-ae92-65378ffd0a55 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.853746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3587704-9f72-46c4-bda5-2a7ba6b79e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.854410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3587704-9f72-46c4-bda5-2a7ba6b79e4c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.854739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3587704-9f72-46c4-bda5-2a7ba6b79e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.855014 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3587704-9f72-46c4-bda5-2a7ba6b79e4c None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.855802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3587704-9f72-46c4-bda5-2a7ba6b79e4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:27.856111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3587704-9f72-46c4-bda5-2a7ba6b79e4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.857603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3587704-9f72-46c4-bda5-2a7ba6b79e4c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.869605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 283/588] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:27.870428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3587704-9f72-46c4-bda5-2a7ba6b79e4c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:27.916089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:27.933941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd3471-1239-4103-ae92-65378ffd0a55 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.973471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6caf1-996e-4011-a8c2-82a2e54da40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:27.974132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6caf1-996e-4011-a8c2-82a2e54da40b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:27.974434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6caf1-996e-4011-a8c2-82a2e54da40b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:27.974688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6caf1-996e-4011-a8c2-82a2e54da40b None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:27.975409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6caf1-996e-4011-a8c2-82a2e54da40b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:27.975718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6caf1-996e-4011-a8c2-82a2e54da40b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:27.975970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:27.977240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6caf1-996e-4011-a8c2-82a2e54da40b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:27.979620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3587704-9f72-46c4-bda5-2a7ba6b79e4c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:27.989528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 306/589] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:27.989528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6caf1-996e-4011-a8c2-82a2e54da40b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.019265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bb661-8076-4dc6-a8b1-c075f8c03b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.020081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bb661-8076-4dc6-a8b1-c075f8c03b71 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.020352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bb661-8076-4dc6-a8b1-c075f8c03b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.020632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bb661-8076-4dc6-a8b1-c075f8c03b71 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.029308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bb661-8076-4dc6-a8b1-c075f8c03b71 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.030769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bb661-8076-4dc6-a8b1-c075f8c03b71 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-ed3bb661-8076-4dc6-a8b1-c075f8c03b71 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:28.045412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:28.046454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:28.047360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 284/590] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:28.047939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bb661-8076-4dc6-a8b1-c075f8c03b71 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.103875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6caf1-996e-4011-a8c2-82a2e54da40b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:28.144923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95b6ab-0948-43ad-80ae-e73ce662b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.146017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95b6ab-0948-43ad-80ae-e73ce662b222 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.146300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95b6ab-0948-43ad-80ae-e73ce662b222 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.146573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95b6ab-0948-43ad-80ae-e73ce662b222 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.154876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95b6ab-0948-43ad-80ae-e73ce662b222 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.156328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95b6ab-0948-43ad-80ae-e73ce662b222 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.169182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bb661-8076-4dc6-a8b1-c075f8c03b71 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-7e95b6ab-0948-43ad-80ae-e73ce662b222 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:28.170580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:28.171625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:28.172525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:28.172525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:28.172525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:28.172525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:28.172525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:28.172525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:28.172525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:28.172525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:28.172525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 307/591] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:28.173027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95b6ab-0948-43ad-80ae-e73ce662b222 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.180888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:28.211794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b486e95-3ae4-4d3c-9e0a-88cb7de6ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.212784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486e95-3ae4-4d3c-9e0a-88cb7de6ef11 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.213161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486e95-3ae4-4d3c-9e0a-88cb7de6ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.213440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486e95-3ae4-4d3c-9e0a-88cb7de6ef11 None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/fb0e53d4b3774959820ac72824b3d4c2/roles/2c4cf4c4707947848a30121234a5157c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.228439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b486e95-3ae4-4d3c-9e0a-88cb7de6ef11 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=fb0e53d4b3774959820ac72824b3d4c2, role_id=2c4cf4c4707947848a30121234a5157c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.229933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b486e95-3ae4-4d3c-9e0a-88cb7de6ef11 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.243378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 285/592] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:16:28 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/fb0e53d4b3774959820ac72824b3d4c2/roles/2c4cf4c4707947848a30121234a5157c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:28.244182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b486e95-3ae4-4d3c-9e0a-88cb7de6ef11 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.265477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:28.302348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95b6ab-0948-43ad-80ae-e73ce662b222 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:28.341425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0ecaa-0cf6-432d-9913-362b48d4152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.342248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0ecaa-0cf6-432d-9913-362b48d4152c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.342525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0ecaa-0cf6-432d-9913-362b48d4152c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.342796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0ecaa-0cf6-432d-9913-362b48d4152c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.351795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0ecaa-0cf6-432d-9913-362b48d4152c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.353299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0ecaa-0cf6-432d-9913-362b48d4152c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-59d0ecaa-0cf6-432d-9913-362b48d4152c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:28.371894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:28.373086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:28.374003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 308/593] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:28.375212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0ecaa-0cf6-432d-9913-362b48d4152c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.384853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b486e95-3ae4-4d3c-9e0a-88cb7de6ef11 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:28.425869 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015610b-2592-497c-9177-e6f3ba4d120a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.426482 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015610b-2592-497c-9177-e6f3ba4d120a None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.426747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015610b-2592-497c-9177-e6f3ba4d120a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.427020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015610b-2592-497c-9177-e6f3ba4d120a None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.427875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015610b-2592-497c-9177-e6f3ba4d120a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:28.428184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015610b-2592-497c-9177-e6f3ba4d120a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.429737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015610b-2592-497c-9177-e6f3ba4d120a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.446465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:28.451083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 286/594] 10.222.0.30 () {56 vars in 1139 bytes} [Mon May 23 15:16:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:28.454849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd734537-0e18-464e-8c36-f7eb4bf6b59b None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.455190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd734537-0e18-464e-8c36-f7eb4bf6b59b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.455475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd734537-0e18-464e-8c36-f7eb4bf6b59b None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.456974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 287/595] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:28.457577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd734537-0e18-464e-8c36-f7eb4bf6b59b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.491576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0ecaa-0cf6-432d-9913-362b48d4152c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:28.529068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146856f3-da02-4ed0-b27c-7dd3328fc42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.530038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146856f3-da02-4ed0-b27c-7dd3328fc42e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.530322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146856f3-da02-4ed0-b27c-7dd3328fc42e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.530600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146856f3-da02-4ed0-b27c-7dd3328fc42e None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.531385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146856f3-da02-4ed0-b27c-7dd3328fc42e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:28.531692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146856f3-da02-4ed0-b27c-7dd3328fc42e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.533152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146856f3-da02-4ed0-b27c-7dd3328fc42e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.536343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:28.546149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 309/596] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:28.546816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146856f3-da02-4ed0-b27c-7dd3328fc42e None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.581829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd734537-0e18-464e-8c36-f7eb4bf6b59b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:28.622492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2723ac3-4157-4021-9cd4-d251765bcef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.623238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2723ac3-4157-4021-9cd4-d251765bcef2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.623523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2723ac3-4157-4021-9cd4-d251765bcef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.623796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2723ac3-4157-4021-9cd4-d251765bcef2 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.624629 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2723ac3-4157-4021-9cd4-d251765bcef2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:28.624984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2723ac3-4157-4021-9cd4-d251765bcef2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.626430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2723ac3-4157-4021-9cd4-d251765bcef2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.643983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 288/597] 10.222.0.30 () {56 vars in 1137 bytes} [Mon May 23 15:16:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:28.645107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f05113-a5d8-462a-9055-ae327be0ee6f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.645392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f05113-a5d8-462a-9055-ae327be0ee6f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.645662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f05113-a5d8-462a-9055-ae327be0ee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.661321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146856f3-da02-4ed0-b27c-7dd3328fc42e None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:28.662434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-02f05113-a5d8-462a-9055-ae327be0ee6f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:28.663592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-02f05113-a5d8-462a-9055-ae327be0ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:28.689737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:28.702808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b62d60-cb10-46fd-bbd8-b14d8dfee0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.703610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b62d60-cb10-46fd-bbd8-b14d8dfee0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.703896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b62d60-cb10-46fd-bbd8-b14d8dfee0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.704176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b62d60-cb10-46fd-bbd8-b14d8dfee0d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.715262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b62d60-cb10-46fd-bbd8-b14d8dfee0d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.716796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b62d60-cb10-46fd-bbd8-b14d8dfee0d5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-60b62d60-cb10-46fd-bbd8-b14d8dfee0d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:28.736061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:28.737037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:28.737908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:28.737908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:28.737908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:28.737908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:28.737908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 310/598] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:28.738176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b62d60-cb10-46fd-bbd8-b14d8dfee0d5 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.745136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 289/599] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:28.745136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f05113-a5d8-462a-9055-ae327be0ee6f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.817229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:28.847910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b62d60-cb10-46fd-bbd8-b14d8dfee0d5 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:28.867891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f05113-a5d8-462a-9055-ae327be0ee6f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:28.884051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137810b-9242-4849-98dc-55045f53b148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.885217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137810b-9242-4849-98dc-55045f53b148 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.885656 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137810b-9242-4849-98dc-55045f53b148 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.885978 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137810b-9242-4849-98dc-55045f53b148 None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/232cfaefaf584ac59720713e02cf2c2d/roles/2c4cf4c4707947848a30121234a5157c` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.901202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137810b-9242-4849-98dc-55045f53b148 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=232cfaefaf584ac59720713e02cf2c2d, role_id=2c4cf4c4707947848a30121234a5157c)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.902629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137810b-9242-4849-98dc-55045f53b148 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.904287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819371a-0846-44ac-8b2f-06a9b349c3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:28.905099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819371a-0846-44ac-8b2f-06a9b349c3cc None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:28.905375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819371a-0846-44ac-8b2f-06a9b349c3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:28.905879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819371a-0846-44ac-8b2f-06a9b349c3cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:28.916728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 311/600] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:16:28 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/232cfaefaf584ac59720713e02cf2c2d/roles/2c4cf4c4707947848a30121234a5157c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:28.917149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819371a-0846-44ac-8b2f-06a9b349c3cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:28.918206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137810b-9242-4849-98dc-55045f53b148 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.918544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819371a-0846-44ac-8b2f-06a9b349c3cc None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-7819371a-0846-44ac-8b2f-06a9b349c3cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:28.946831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:28.947787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:28.948742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:28.948742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:28.948742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:28.948742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:28.948742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 290/601] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:28.949543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819371a-0846-44ac-8b2f-06a9b349c3cc None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:28.963173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:29.067001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137810b-9242-4849-98dc-55045f53b148 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.088450 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819371a-0846-44ac-8b2f-06a9b349c3cc None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.097899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:29.129297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6cf14-3529-4243-9e31-c9b9261c13d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.129996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f573964-1592-4aee-b9d9-fb17ae55b5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.130580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6cf14-3529-4243-9e31-c9b9261c13d0 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.130781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573964-1592-4aee-b9d9-fb17ae55b5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.131111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573964-1592-4aee-b9d9-fb17ae55b5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.131382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573964-1592-4aee-b9d9-fb17ae55b5f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.131692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6cf14-3529-4243-9e31-c9b9261c13d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.132146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6cf14-3529-4243-9e31-c9b9261c13d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.134149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6cf14-3529-4243-9e31-c9b9261c13d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:29.135088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6cf14-3529-4243-9e31-c9b9261c13d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:29.136645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6cf14-3529-4243-9e31-c9b9261c13d0 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:29.144365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f573964-1592-4aee-b9d9-fb17ae55b5f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:29.150628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 312/602] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:29.151668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6cf14-3529-4243-9e31-c9b9261c13d0 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-9f573964-1592-4aee-b9d9-fb17ae55b5f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:29.156383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:29.157357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 291/603] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:29.158215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f573964-1592-4aee-b9d9-fb17ae55b5f3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:29.270129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:29.271587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f573964-1592-4aee-b9d9-fb17ae55b5f3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.288197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6cf14-3529-4243-9e31-c9b9261c13d0 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.307819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db6b47-a7a0-4bda-9f49-43d17b5c3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.308624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db6b47-a7a0-4bda-9f49-43d17b5c3b8b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.308933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db6b47-a7a0-4bda-9f49-43d17b5c3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.309214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db6b47-a7a0-4bda-9f49-43d17b5c3b8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.316450 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db6b47-a7a0-4bda-9f49-43d17b5c3b8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:29.317966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db6b47-a7a0-4bda-9f49-43d17b5c3b8b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:29.331326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f0ee2-a466-44ea-9f30-8125eac63db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.332238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f0ee2-a466-44ea-9f30-8125eac63db1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.332682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f0ee2-a466-44ea-9f30-8125eac63db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.333126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f0ee2-a466-44ea-9f30-8125eac63db1 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.333984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f0ee2-a466-44ea-9f30-8125eac63db1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:29.334425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f0ee2-a466-44ea-9f30-8125eac63db1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-52db6b47-a7a0-4bda-9f49-43d17b5c3b8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:29.334677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:29.335681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:29.337181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:29.337181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:29.337181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:29.337181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:29.337181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:29.337181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:29.337181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:29.337181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:29.337181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 292/604] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:29.337696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db6b47-a7a0-4bda-9f49-43d17b5c3b8b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:29.339090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f0ee2-a466-44ea-9f30-8125eac63db1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:29.355918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 313/605] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:29.357006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f0ee2-a466-44ea-9f30-8125eac63db1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:29.372432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:29.468063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db6b47-a7a0-4bda-9f49-43d17b5c3b8b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.498791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f0ee2-a466-44ea-9f30-8125eac63db1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.504782 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffcb96-5f31-4878-b047-95024cd21655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.505524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffcb96-5f31-4878-b047-95024cd21655 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.505807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffcb96-5f31-4878-b047-95024cd21655 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.506079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffcb96-5f31-4878-b047-95024cd21655 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.506828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffcb96-5f31-4878-b047-95024cd21655 None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:29.507132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffcb96-5f31-4878-b047-95024cd21655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:29.509288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffcb96-5f31-4878-b047-95024cd21655 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:29.517979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 293/606] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:29.518809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffcb96-5f31-4878-b047-95024cd21655 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:29.546035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317f660-c090-4be0-b0ed-b7ef27f25ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.546929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317f660-c090-4be0-b0ed-b7ef27f25ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.547356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317f660-c090-4be0-b0ed-b7ef27f25ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.547773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317f660-c090-4be0-b0ed-b7ef27f25ee9 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.549252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317f660-c090-4be0-b0ed-b7ef27f25ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:29.549872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317f660-c090-4be0-b0ed-b7ef27f25ee9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:29.552271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317f660-c090-4be0-b0ed-b7ef27f25ee9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:29.568800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 314/607] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:29.569847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317f660-c090-4be0-b0ed-b7ef27f25ee9 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:29.581889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:29.622693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:29.637184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffcb96-5f31-4878-b047-95024cd21655 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.674879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfca57d-d3c8-44ac-892d-7d1fa005dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.675813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfca57d-d3c8-44ac-892d-7d1fa005dcdc None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.676193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfca57d-d3c8-44ac-892d-7d1fa005dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.676565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfca57d-d3c8-44ac-892d-7d1fa005dcdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.686554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfca57d-d3c8-44ac-892d-7d1fa005dcdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:29.688385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfca57d-d3c8-44ac-892d-7d1fa005dcdc None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-ddfca57d-d3c8-44ac-892d-7d1fa005dcdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:29.706966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:29.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:29.709140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:29.709140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:29.709140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:29.709140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:29.709140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:29.709140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:29.709140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:29.709140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:29.711166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317f660-c090-4be0-b0ed-b7ef27f25ee9 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.715460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 294/608] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:29.716410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfca57d-d3c8-44ac-892d-7d1fa005dcdc None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:29.756885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ada2b-4a35-4e6a-b24b-38ad09040761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.758581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ada2b-4a35-4e6a-b24b-38ad09040761 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.759014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ada2b-4a35-4e6a-b24b-38ad09040761 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.759433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ada2b-4a35-4e6a-b24b-38ad09040761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.773989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ada2b-4a35-4e6a-b24b-38ad09040761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:29.776164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ada2b-4a35-4e6a-b24b-38ad09040761 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-dc5ada2b-4a35-4e6a-b24b-38ad09040761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:29.799562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:29.800618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:29.801688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:29.801688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:29.801688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:29.801688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:29.808324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 315/609] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:29.809377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ada2b-4a35-4e6a-b24b-38ad09040761 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:29.839326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfca57d-d3c8-44ac-892d-7d1fa005dcdc None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.871439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:29.886376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafea4b-ebe9-4484-8a65-d4990e9c2853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.887523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafea4b-ebe9-4484-8a65-d4990e9c2853 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.888489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafea4b-ebe9-4484-8a65-d4990e9c2853 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.888489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafea4b-ebe9-4484-8a65-d4990e9c2853 None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/68d85e20dc884cd0bad5bb6c304776ca/roles/2c4cf4c4707947848a30121234a5157c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.904427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafea4b-ebe9-4484-8a65-d4990e9c2853 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=68d85e20dc884cd0bad5bb6c304776ca, role_id=2c4cf4c4707947848a30121234a5157c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:29.906117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafea4b-ebe9-4484-8a65-d4990e9c2853 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:29.916443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:29.923015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 295/610] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:16:29 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/68d85e20dc884cd0bad5bb6c304776ca/roles/2c4cf4c4707947848a30121234a5157c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:29.924222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafea4b-ebe9-4484-8a65-d4990e9c2853 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:29.929566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ada2b-4a35-4e6a-b24b-38ad09040761 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:29.970091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e06f6-b110-472c-9f17-697cf6866d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:29.971057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e06f6-b110-472c-9f17-697cf6866d56 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:29.971487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e06f6-b110-472c-9f17-697cf6866d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:29.971915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e06f6-b110-472c-9f17-697cf6866d56 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:29.972904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e06f6-b110-472c-9f17-697cf6866d56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:29.973393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e06f6-b110-472c-9f17-697cf6866d56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:29.975082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e06f6-b110-472c-9f17-697cf6866d56 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:29.989627 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 316/611] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:29.995555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e06f6-b110-472c-9f17-697cf6866d56 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:30.045171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafea4b-ebe9-4484-8a65-d4990e9c2853 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:30.085336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbca7cb-4c90-4d60-ae91-0cffdf1085d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:30.086526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbca7cb-4c90-4d60-ae91-0cffdf1085d2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:30.086920 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbca7cb-4c90-4d60-ae91-0cffdf1085d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:30.087429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbca7cb-4c90-4d60-ae91-0cffdf1085d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:30.088375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbca7cb-4c90-4d60-ae91-0cffdf1085d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:30.088756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbca7cb-4c90-4d60-ae91-0cffdf1085d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:30.090545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbca7cb-4c90-4d60-ae91-0cffdf1085d2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:30.104826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 296/612] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:30.110577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbca7cb-4c90-4d60-ae91-0cffdf1085d2 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:30.110970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e06f6-b110-472c-9f17-697cf6866d56 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:30.131603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:30.151112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db839a-3269-41f1-a376-8867e98dc58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:30.152361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db839a-3269-41f1-a376-8867e98dc58a None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:30.152816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db839a-3269-41f1-a376-8867e98dc58a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:30.153351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db839a-3269-41f1-a376-8867e98dc58a None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/36daf80a4cbc4cfc8962c7924433d257/roles/2c4cf4c4707947848a30121234a5157c` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:30.172353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db839a-3269-41f1-a376-8867e98dc58a None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=36daf80a4cbc4cfc8962c7924433d257, role_id=2c4cf4c4707947848a30121234a5157c)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:30.174118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db839a-3269-41f1-a376-8867e98dc58a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:30.183222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:30.189891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 317/613] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:16:29 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/36daf80a4cbc4cfc8962c7924433d257/roles/2c4cf4c4707947848a30121234a5157c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:16:30.236237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbca7cb-4c90-4d60-ae91-0cffdf1085d2 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:30.279493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70deea5-ca6b-4829-9d9d-1fc05c375f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:30.280009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70deea5-ca6b-4829-9d9d-1fc05c375f3e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:30.280352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70deea5-ca6b-4829-9d9d-1fc05c375f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:30.280635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70deea5-ca6b-4829-9d9d-1fc05c375f3e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:30.290674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70deea5-ca6b-4829-9d9d-1fc05c375f3e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:30.292163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70deea5-ca6b-4829-9d9d-1fc05c375f3e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-c70deea5-ca6b-4829-9d9d-1fc05c375f3e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:30.308593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:30.309673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:30.310573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 297/614] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:30.318031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db839a-3269-41f1-a376-8867e98dc58a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:30.444621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db839a-3269-41f1-a376-8867e98dc58a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:30.482308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674934d0-eb83-40a5-9829-376791a85071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:30.483318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674934d0-eb83-40a5-9829-376791a85071 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:30.483782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674934d0-eb83-40a5-9829-376791a85071 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:30.484287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674934d0-eb83-40a5-9829-376791a85071 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:30.485376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674934d0-eb83-40a5-9829-376791a85071 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:30.485866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674934d0-eb83-40a5-9829-376791a85071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:30.487673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674934d0-eb83-40a5-9829-376791a85071 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:30.508217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 318/615] 10.222.0.30 () {56 vars in 1135 bytes} [Mon May 23 15:16:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:30.513975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70deea5-ca6b-4829-9d9d-1fc05c375f3e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:30.550827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93780f1-70ce-43f2-8958-0128750a0e89 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:30.551400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93780f1-70ce-43f2-8958-0128750a0e89 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:30.551892 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93780f1-70ce-43f2-8958-0128750a0e89 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:30.553854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 319/616] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:30.565505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13949b-32c5-46d1-8c7a-ed472cedf29f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:30.565795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13949b-32c5-46d1-8c7a-ed472cedf29f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:30.566079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13949b-32c5-46d1-8c7a-ed472cedf29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:30.587683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0c13949b-32c5-46d1-8c7a-ed472cedf29f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:30.589214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0c13949b-32c5-46d1-8c7a-ed472cedf29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:30.616924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:30.674734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 320/617] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:30.698871 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13949b-32c5-46d1-8c7a-ed472cedf29f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:30.699399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70deea5-ca6b-4829-9d9d-1fc05c375f3e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:30.729230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:30.761708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a0434-e03c-430a-9a11-4ddcbb2ba090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:30.762509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a0434-e03c-430a-9a11-4ddcbb2ba090 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:30.762853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a0434-e03c-430a-9a11-4ddcbb2ba090 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:30.763128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a0434-e03c-430a-9a11-4ddcbb2ba090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:30.780753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a0434-e03c-430a-9a11-4ddcbb2ba090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:30.782256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a0434-e03c-430a-9a11-4ddcbb2ba090 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-363a0434-e03c-430a-9a11-4ddcbb2ba090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:30.805547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:30.806518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:30.807375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:30.807375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:30.807375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:30.807375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:30.807375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 298/618] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:30.812398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a0434-e03c-430a-9a11-4ddcbb2ba090 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:30.869010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13949b-32c5-46d1-8c7a-ed472cedf29f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:30.932445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e35379-6607-4155-b516-15827b028839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:30.933466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e35379-6607-4155-b516-15827b028839 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:30.933644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e35379-6607-4155-b516-15827b028839 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:30.936896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e35379-6607-4155-b516-15827b028839 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:30.949452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e35379-6607-4155-b516-15827b028839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-75e35379-6607-4155-b516-15827b028839 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:30.957228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:30.959458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:30.962242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:30.970082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 321/619] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:30.973566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a0434-e03c-430a-9a11-4ddcbb2ba090 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:30.980926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e35379-6607-4155-b516-15827b028839 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:31.014981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:31.020900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f8154-3f58-4a4c-ad36-120e6c45bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:31.021575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8154-3f58-4a4c-ad36-120e6c45bf79 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:31.021856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8154-3f58-4a4c-ad36-120e6c45bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:31.022132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8154-3f58-4a4c-ad36-120e6c45bf79 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:31.024861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f8154-3f58-4a4c-ad36-120e6c45bf79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:31.025234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f8154-3f58-4a4c-ad36-120e6c45bf79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:31.026639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f8154-3f58-4a4c-ad36-120e6c45bf79 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:31.061042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 299/620] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:31.064643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f8154-3f58-4a4c-ad36-120e6c45bf79 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:31.125191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:31.170232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e35379-6607-4155-b516-15827b028839 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:31.237663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32aa00-61bc-40a9-b42b-0a0c4d364387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:31.238264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32aa00-61bc-40a9-b42b-0a0c4d364387 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:31.238544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32aa00-61bc-40a9-b42b-0a0c4d364387 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:31.238819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32aa00-61bc-40a9-b42b-0a0c4d364387 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:31.239572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32aa00-61bc-40a9-b42b-0a0c4d364387 None admin] RBAC: Adding query filter params (name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:31.239880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32aa00-61bc-40a9-b42b-0a0c4d364387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:31.241291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32aa00-61bc-40a9-b42b-0a0c4d364387 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:31.253793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f8154-3f58-4a4c-ad36-120e6c45bf79 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:31.258272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 322/621] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:31.268553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32aa00-61bc-40a9-b42b-0a0c4d364387 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:31.318141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebaa9e-3fb2-416e-9975-652af6380321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:31.318998 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebaa9e-3fb2-416e-9975-652af6380321 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:31.319277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebaa9e-3fb2-416e-9975-652af6380321 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:31.319548 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebaa9e-3fb2-416e-9975-652af6380321 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:31.336663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ebaa9e-3fb2-416e-9975-652af6380321 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:31.338175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ebaa9e-3fb2-416e-9975-652af6380321 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-96ebaa9e-3fb2-416e-9975-652af6380321 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:31.367602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:31.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:31.369531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:31.369531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:31.369531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:31.369531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:31.369531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:31.369531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:31.369531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:31.369531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:31.374504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 300/622] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:31.378800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebaa9e-3fb2-416e-9975-652af6380321 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:31.421228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:31.459969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32aa00-61bc-40a9-b42b-0a0c4d364387 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:31.505022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86db77c-00fd-4930-8f38-723506fc088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:31.505753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86db77c-00fd-4930-8f38-723506fc088f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:31.506369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86db77c-00fd-4930-8f38-723506fc088f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:31.506653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86db77c-00fd-4930-8f38-723506fc088f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:31.520334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86db77c-00fd-4930-8f38-723506fc088f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:31.522726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86db77c-00fd-4930-8f38-723506fc088f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-a86db77c-00fd-4930-8f38-723506fc088f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:31.543462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:31.544415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:31.545383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:31.545383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:31.545383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:31.545383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:31.549354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 323/623] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:31.551799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86db77c-00fd-4930-8f38-723506fc088f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:31.574640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:31.602106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebaa9e-3fb2-416e-9975-652af6380321 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:31.652224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb026de8-fdaf-4582-855e-5324b97b41bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:31.653114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb026de8-fdaf-4582-855e-5324b97b41bf None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:31.653501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb026de8-fdaf-4582-855e-5324b97b41bf None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:31.653897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb026de8-fdaf-4582-855e-5324b97b41bf None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:31.654911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb026de8-fdaf-4582-855e-5324b97b41bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:31.655417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb026de8-fdaf-4582-855e-5324b97b41bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:31.656996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb026de8-fdaf-4582-855e-5324b97b41bf None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:31.665224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86db77c-00fd-4930-8f38-723506fc088f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:31.676095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 301/624] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:31.677811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd171-2121-4ad3-96a8-ac3fe5a328ea None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:31.678196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd171-2121-4ad3-96a8-ac3fe5a328ea None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:31.678568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd171-2121-4ad3-96a8-ac3fe5a328ea None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:31.684506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 302/625] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:31.684506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecd171-2121-4ad3-96a8-ac3fe5a328ea None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:31.704846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:31.718459 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648ef46-cde5-4b4d-9651-68fe12fafd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:31.719998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648ef46-cde5-4b4d-9651-68fe12fafd05 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:31.720728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648ef46-cde5-4b4d-9651-68fe12fafd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:31.721202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648ef46-cde5-4b4d-9651-68fe12fafd05 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:31.722385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648ef46-cde5-4b4d-9651-68fe12fafd05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:31.722867 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648ef46-cde5-4b4d-9651-68fe12fafd05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:31.725271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648ef46-cde5-4b4d-9651-68fe12fafd05 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:31.742756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 324/626] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:31.744416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8af272-5b00-494d-999b-87d1e9186f78 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:31.744925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8af272-5b00-494d-999b-87d1e9186f78 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:31.745401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8af272-5b00-494d-999b-87d1e9186f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:31.767812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-7b8af272-5b00-494d-999b-87d1e9186f78 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:31.769589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7b8af272-5b00-494d-999b-87d1e9186f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:31.811139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecd171-2121-4ad3-96a8-ac3fe5a328ea None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:31.849862 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577df0b4-9452-4710-bb2a-64003bfc9ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:31.850808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577df0b4-9452-4710-bb2a-64003bfc9ebc None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:31.851108 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577df0b4-9452-4710-bb2a-64003bfc9ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:31.851391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577df0b4-9452-4710-bb2a-64003bfc9ebc None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/ff6c489d950a4370bbf01866d2fc8fc3/roles/2c4cf4c4707947848a30121234a5157c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:31.863504 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 325/627] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:31.864614 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8af272-5b00-494d-999b-87d1e9186f78 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:31.875444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577df0b4-9452-4710-bb2a-64003bfc9ebc None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=ff6c489d950a4370bbf01866d2fc8fc3, role_id=2c4cf4c4707947848a30121234a5157c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:31.877263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577df0b4-9452-4710-bb2a-64003bfc9ebc None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:31.888326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:31.897036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 303/628] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:16:31 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/ff6c489d950a4370bbf01866d2fc8fc3/roles/2c4cf4c4707947848a30121234a5157c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:31.911047 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577df0b4-9452-4710-bb2a-64003bfc9ebc None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:32.030127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8af272-5b00-494d-999b-87d1e9186f78 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:32.057502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:32.075343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167661e-18e1-4944-997b-875247711742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:32.076552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167661e-18e1-4944-997b-875247711742 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:32.077109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167661e-18e1-4944-997b-875247711742 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:32.077592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167661e-18e1-4944-997b-875247711742 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:32.088392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167661e-18e1-4944-997b-875247711742 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:32.091116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167661e-18e1-4944-997b-875247711742 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-b167661e-18e1-4944-997b-875247711742 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:32.109802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:32.110810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 15:16:32.111851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:32.114839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 326/629] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:32.122296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167661e-18e1-4944-997b-875247711742 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:32.139580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577df0b4-9452-4710-bb2a-64003bfc9ebc None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:32.187400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcc319-9707-4c6f-899d-ee4b347fcd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:32.188206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcc319-9707-4c6f-899d-ee4b347fcd40 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:32.188486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcc319-9707-4c6f-899d-ee4b347fcd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:32.188750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcc319-9707-4c6f-899d-ee4b347fcd40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:32.205583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcc319-9707-4c6f-899d-ee4b347fcd40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-54bcc319-9707-4c6f-899d-ee4b347fcd40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:32.214915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:32.215877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:32.216714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 304/630] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:32.225679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcc319-9707-4c6f-899d-ee4b347fcd40 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:32.274712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167661e-18e1-4944-997b-875247711742 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:32.345370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:32.361683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43911844-5d48-499f-a223-a1353220c907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:32.362919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911844-5d48-499f-a223-a1353220c907 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:32.364001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911844-5d48-499f-a223-a1353220c907 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:32.364595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911844-5d48-499f-a223-a1353220c907 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:32.366615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911844-5d48-499f-a223-a1353220c907 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:32.367310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911844-5d48-499f-a223-a1353220c907 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:32.370346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911844-5d48-499f-a223-a1353220c907 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:32.381398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcc319-9707-4c6f-899d-ee4b347fcd40 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:32.404659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 327/631] 10.222.0.30 () {56 vars in 1137 bytes} [Mon May 23 15:16:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:32.410793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43911844-5d48-499f-a223-a1353220c907 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:32.444612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a0761-1eea-47cc-aa7e-9aded887b667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:32.448070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a0761-1eea-47cc-aa7e-9aded887b667 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:32.448513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a0761-1eea-47cc-aa7e-9aded887b667 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:32.448981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a0761-1eea-47cc-aa7e-9aded887b667 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:32.449969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a0761-1eea-47cc-aa7e-9aded887b667 None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:32.450425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a0761-1eea-47cc-aa7e-9aded887b667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:32.451983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a0761-1eea-47cc-aa7e-9aded887b667 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:32.470700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 305/632] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:32.473816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a0761-1eea-47cc-aa7e-9aded887b667 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:32.485175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:32.586861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43911844-5d48-499f-a223-a1353220c907 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:32.651297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca205a9c-1998-478d-8c94-3b18dd89b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:32.651974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca205a9c-1998-478d-8c94-3b18dd89b838 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:32.652257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca205a9c-1998-478d-8c94-3b18dd89b838 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:32.652532 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca205a9c-1998-478d-8c94-3b18dd89b838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:32.687649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a0761-1eea-47cc-aa7e-9aded887b667 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:32.730087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:32.754531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e55bb-7bbf-465d-957e-2b0a3b2b4d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:32.755422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e55bb-7bbf-465d-957e-2b0a3b2b4d34 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:32.755704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e55bb-7bbf-465d-957e-2b0a3b2b4d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:32.756272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e55bb-7bbf-465d-957e-2b0a3b2b4d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:32.778022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e55bb-7bbf-465d-957e-2b0a3b2b4d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:32.779520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e55bb-7bbf-465d-957e-2b0a3b2b4d34 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:32.784806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca205a9c-1998-478d-8c94-3b18dd89b838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:32.786297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca205a9c-1998-478d-8c94-3b18dd89b838 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-8b0e55bb-7bbf-465d-957e-2b0a3b2b4d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:32.811699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:32.813153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:32.814316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:32.814316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:32.814316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:32.814316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:32.814316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 306/633] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:32.814659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41ec49-16ce-4d83-b3fe-82bdbbe5456f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:32.814776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41ec49-16ce-4d83-b3fe-82bdbbe5456f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:32.815051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41ec49-16ce-4d83-b3fe-82bdbbe5456f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:32.825695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 307/634] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:32.825695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41ec49-16ce-4d83-b3fe-82bdbbe5456f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:32.848653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:32.937239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 328/635] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:32.938824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e86259-6fd7-46d4-b3eb-94e036274a40 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:32.939120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e86259-6fd7-46d4-b3eb-94e036274a40 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:32.939422 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e86259-6fd7-46d4-b3eb-94e036274a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:32.951483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41ec49-16ce-4d83-b3fe-82bdbbe5456f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:32.958669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-94e86259-6fd7-46d4-b3eb-94e036274a40 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:32.960011 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-94e86259-6fd7-46d4-b3eb-94e036274a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:32.992841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dee48c-58ca-4503-a246-6fae010d2814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:32.993558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dee48c-58ca-4503-a246-6fae010d2814 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:32.993864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dee48c-58ca-4503-a246-6fae010d2814 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:32.994168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dee48c-58ca-4503-a246-6fae010d2814 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:32.994944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dee48c-58ca-4503-a246-6fae010d2814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:32.995377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dee48c-58ca-4503-a246-6fae010d2814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:32.997127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dee48c-58ca-4503-a246-6fae010d2814 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:33.010284 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 308/636] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:33.011129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dee48c-58ca-4503-a246-6fae010d2814 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.041247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 329/637] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:33.043260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31569bb1-d498-4dfe-b590-045e21f75b6f None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.043735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31569bb1-d498-4dfe-b590-045e21f75b6f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.044180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31569bb1-d498-4dfe-b590-045e21f75b6f None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.045967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 330/638] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:33.046864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31569bb1-d498-4dfe-b590-045e21f75b6f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.053031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:33.123658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:33.129345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dee48c-58ca-4503-a246-6fae010d2814 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.158189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31569bb1-d498-4dfe-b590-045e21f75b6f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.166427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d07733-518f-4c8a-80e3-49ff448a0db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.167207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d07733-518f-4c8a-80e3-49ff448a0db2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.167490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d07733-518f-4c8a-80e3-49ff448a0db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.167787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d07733-518f-4c8a-80e3-49ff448a0db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.178996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d07733-518f-4c8a-80e3-49ff448a0db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:33.181366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d07733-518f-4c8a-80e3-49ff448a0db2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:33.194657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e623c-e925-41d7-9039-6304f94a63bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.195487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e623c-e925-41d7-9039-6304f94a63bd None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.195787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e623c-e925-41d7-9039-6304f94a63bd None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.196063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e623c-e925-41d7-9039-6304f94a63bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-f4d07733-518f-4c8a-80e3-49ff448a0db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:33.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:33.200711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:33.201871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:33.201871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:33.201871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:33.201871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:33.201871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 309/639] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:33.202574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20d22e-e348-4dfd-92ac-d350552f7a85 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.202857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20d22e-e348-4dfd-92ac-d350552f7a85 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.203134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20d22e-e348-4dfd-92ac-d350552f7a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.207938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e623c-e925-41d7-9039-6304f94a63bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:33.209410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e623c-e925-41d7-9039-6304f94a63bd None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:33.219646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ef20d22e-e348-4dfd-92ac-d350552f7a85 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:33.220838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ef20d22e-e348-4dfd-92ac-d350552f7a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-570e623c-e925-41d7-9039-6304f94a63bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:33.223974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:33.225015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:33.225971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 331/640] 10.222.0.30 () {56 vars in 1093 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:33.227107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e623c-e925-41d7-9039-6304f94a63bd None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.305340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 310/641] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:33.305340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20d22e-e348-4dfd-92ac-d350552f7a85 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.322021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:33.337906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e623c-e925-41d7-9039-6304f94a63bd None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.366348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:33.376235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae1fea-a798-417f-89fc-da3d97cdc804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.377066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae1fea-a798-417f-89fc-da3d97cdc804 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.377351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae1fea-a798-417f-89fc-da3d97cdc804 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.377672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae1fea-a798-417f-89fc-da3d97cdc804 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.388458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae1fea-a798-417f-89fc-da3d97cdc804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-35ae1fea-a798-417f-89fc-da3d97cdc804 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:33.396447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:33.397659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 332/642] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:33.399103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae1fea-a798-417f-89fc-da3d97cdc804 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.422569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20d22e-e348-4dfd-92ac-d350552f7a85 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.456268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe3960-53c1-4c4a-8a2f-7fee066d07ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.456850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe3960-53c1-4c4a-8a2f-7fee066d07ec None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.457162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe3960-53c1-4c4a-8a2f-7fee066d07ec None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.457438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe3960-53c1-4c4a-8a2f-7fee066d07ec None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.458159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe3960-53c1-4c4a-8a2f-7fee066d07ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:33.458463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe3960-53c1-4c4a-8a2f-7fee066d07ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:33.459846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe3960-53c1-4c4a-8a2f-7fee066d07ec None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:33.473258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 311/643] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:33.473879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe3960-53c1-4c4a-8a2f-7fee066d07ec None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.512972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae1fea-a798-417f-89fc-da3d97cdc804 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.549084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc750a95-8cfb-4df4-bb0f-67df6d7bead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.550041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc750a95-8cfb-4df4-bb0f-67df6d7bead5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.550499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc750a95-8cfb-4df4-bb0f-67df6d7bead5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.550944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc750a95-8cfb-4df4-bb0f-67df6d7bead5 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.552021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc750a95-8cfb-4df4-bb0f-67df6d7bead5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:33.552511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc750a95-8cfb-4df4-bb0f-67df6d7bead5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:33.554220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc750a95-8cfb-4df4-bb0f-67df6d7bead5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:33.554896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:33.572971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 333/644] 10.222.0.30 () {56 vars in 1133 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:33.578154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc750a95-8cfb-4df4-bb0f-67df6d7bead5 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.579422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe3960-53c1-4c4a-8a2f-7fee066d07ec None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.619407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9061f00-b093-4b77-ba7a-44b345751cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.620278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9061f00-b093-4b77-ba7a-44b345751cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.620631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9061f00-b093-4b77-ba7a-44b345751cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.620929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9061f00-b093-4b77-ba7a-44b345751cd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.631637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9061f00-b093-4b77-ba7a-44b345751cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:33.633551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-d9061f00-b093-4b77-ba7a-44b345751cd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:33.640445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:33.641479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:33.642631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:33.646251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 312/645] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:33.647076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9061f00-b093-4b77-ba7a-44b345751cd6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.693520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc750a95-8cfb-4df4-bb0f-67df6d7bead5 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.730394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c2130-aa6f-400f-a363-b900031eeca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.731406 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c2130-aa6f-400f-a363-b900031eeca3 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.731698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c2130-aa6f-400f-a363-b900031eeca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.731975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c2130-aa6f-400f-a363-b900031eeca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.732731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c2130-aa6f-400f-a363-b900031eeca3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:33.733412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c2130-aa6f-400f-a363-b900031eeca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:33.735108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c2130-aa6f-400f-a363-b900031eeca3 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:33.745070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 334/646] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:33.745717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c2130-aa6f-400f-a363-b900031eeca3 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.763033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9061f00-b093-4b77-ba7a-44b345751cd6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.797672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142973-645a-4b94-9216-1ea6bac19154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.798459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142973-645a-4b94-9216-1ea6bac19154 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.798737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142973-645a-4b94-9216-1ea6bac19154 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.799007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142973-645a-4b94-9216-1ea6bac19154 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.801704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:33.807175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3142973-645a-4b94-9216-1ea6bac19154 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:33.808812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3142973-645a-4b94-9216-1ea6bac19154 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-f3142973-645a-4b94-9216-1ea6bac19154 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:33.823513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:33.825237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 313/647] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:33.826172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142973-645a-4b94-9216-1ea6bac19154 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:33.859523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c2130-aa6f-400f-a363-b900031eeca3 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.870285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:33.897290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c34f84-27e8-4eff-872f-09b341e3b241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.898272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c34f84-27e8-4eff-872f-09b341e3b241 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.898272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c34f84-27e8-4eff-872f-09b341e3b241 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.898717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c34f84-27e8-4eff-872f-09b341e3b241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.935148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142973-645a-4b94-9216-1ea6bac19154 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:33.971911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08697d-12d5-4dcc-ad8a-24d15693061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:33.972580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08697d-12d5-4dcc-ad8a-24d15693061f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:33.972877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08697d-12d5-4dcc-ad8a-24d15693061f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:33.973166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08697d-12d5-4dcc-ad8a-24d15693061f None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:33.973916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08697d-12d5-4dcc-ad8a-24d15693061f None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:33.974222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08697d-12d5-4dcc-ad8a-24d15693061f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:33.975596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08697d-12d5-4dcc-ad8a-24d15693061f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:33.979041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c34f84-27e8-4eff-872f-09b341e3b241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:33.980676 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c34f84-27e8-4eff-872f-09b341e3b241 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:33.985122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 314/648] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:33.985891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08697d-12d5-4dcc-ad8a-24d15693061f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.048669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:34.095587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08697d-12d5-4dcc-ad8a-24d15693061f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:34.100922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 335/649] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:34.100922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c34f84-27e8-4eff-872f-09b341e3b241 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.110174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:34.139703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9724ff-f4a9-4788-bef3-7aeceec04b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:34.140500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9724ff-f4a9-4788-bef3-7aeceec04b6d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.140787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9724ff-f4a9-4788-bef3-7aeceec04b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.141105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9724ff-f4a9-4788-bef3-7aeceec04b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.152631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9724ff-f4a9-4788-bef3-7aeceec04b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:34.154136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9724ff-f4a9-4788-bef3-7aeceec04b6d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-9c9724ff-f4a9-4788-bef3-7aeceec04b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:34.172097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:34.173022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:34.173902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:34.173902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:34.173902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:34.173902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:34.173902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 315/650] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:34.174179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9724ff-f4a9-4788-bef3-7aeceec04b6d None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.215500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c34f84-27e8-4eff-872f-09b341e3b241 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:34.252236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee49cf6-4614-44e9-80e5-9555e279fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:34.252833 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee49cf6-4614-44e9-80e5-9555e279fb2d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.253151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee49cf6-4614-44e9-80e5-9555e279fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.253428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee49cf6-4614-44e9-80e5-9555e279fb2d None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.254235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee49cf6-4614-44e9-80e5-9555e279fb2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:34.254543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee49cf6-4614-44e9-80e5-9555e279fb2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:34.256029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee49cf6-4614-44e9-80e5-9555e279fb2d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:34.275378 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 336/651] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:34.275378 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee49cf6-4614-44e9-80e5-9555e279fb2d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.290534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9724ff-f4a9-4788-bef3-7aeceec04b6d None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:34.301468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:34.329601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88376cab-07e6-44ce-943c-af6b1942943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:34.330505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88376cab-07e6-44ce-943c-af6b1942943e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.330833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88376cab-07e6-44ce-943c-af6b1942943e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.331120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88376cab-07e6-44ce-943c-af6b1942943e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.345336 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:34.347261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88376cab-07e6-44ce-943c-af6b1942943e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:34.349272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88376cab-07e6-44ce-943c-af6b1942943e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-88376cab-07e6-44ce-943c-af6b1942943e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:34.369967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:34.370865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:34.371732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:34.371732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:34.371732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:34.371732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:34.371732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 316/652] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:34.373204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88376cab-07e6-44ce-943c-af6b1942943e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.388534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee49cf6-4614-44e9-80e5-9555e279fb2d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:34.427787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa54104-eb00-4918-9caf-e36543b53263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:34.428948 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa54104-eb00-4918-9caf-e36543b53263 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.429258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa54104-eb00-4918-9caf-e36543b53263 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.429547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa54104-eb00-4918-9caf-e36543b53263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.442508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa54104-eb00-4918-9caf-e36543b53263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:34.444102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa54104-eb00-4918-9caf-e36543b53263 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-9aa54104-eb00-4918-9caf-e36543b53263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:34.462449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:34.463441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:34.464640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:34.464640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:34.464640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:34.464640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:34.464640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 337/653] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:34.464640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa54104-eb00-4918-9caf-e36543b53263 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.484964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88376cab-07e6-44ce-943c-af6b1942943e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:34.519402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ffc94-857f-44f6-9617-e446d7e380a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:34.520090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ffc94-857f-44f6-9617-e446d7e380a0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.520375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ffc94-857f-44f6-9617-e446d7e380a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.520645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ffc94-857f-44f6-9617-e446d7e380a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.521656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ffc94-857f-44f6-9617-e446d7e380a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:34.521772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ffc94-857f-44f6-9617-e446d7e380a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:34.523058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ffc94-857f-44f6-9617-e446d7e380a0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:34.535225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 317/654] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:34.536023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ffc94-857f-44f6-9617-e446d7e380a0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.559236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:34.565806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa54104-eb00-4918-9caf-e36543b53263 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:34.585562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:34.602207 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a410cc-7dfb-4575-bc17-55d30be9967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:34.602866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a410cc-7dfb-4575-bc17-55d30be9967f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.603177 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a410cc-7dfb-4575-bc17-55d30be9967f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.603457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a410cc-7dfb-4575-bc17-55d30be9967f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.604284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a410cc-7dfb-4575-bc17-55d30be9967f None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=fb0e53d4b3774959820ac72824b3d4c2) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:34.604595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a410cc-7dfb-4575-bc17-55d30be9967f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:34.606060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a410cc-7dfb-4575-bc17-55d30be9967f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:34.612141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 338/655] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/role_assignments?user.id=fb0e53d4b3774959820ac72824b3d4c2&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:34.613479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecb732-a1da-4ba3-9982-d5c96440b65c None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.613770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecb732-a1da-4ba3-9982-d5c96440b65c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.614045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecb732-a1da-4ba3-9982-d5c96440b65c None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.615269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 339/656] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:34.615968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecb732-a1da-4ba3-9982-d5c96440b65c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.647733 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ffc94-857f-44f6-9617-e446d7e380a0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:34.686441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882a571-29af-4008-949e-6a31e89248e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:34.687246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882a571-29af-4008-949e-6a31e89248e5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.687598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882a571-29af-4008-949e-6a31e89248e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.687943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882a571-29af-4008-949e-6a31e89248e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.688767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882a571-29af-4008-949e-6a31e89248e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:34.689195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882a571-29af-4008-949e-6a31e89248e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:34.690670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882a571-29af-4008-949e-6a31e89248e5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:34.708164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 318/657] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:34.708164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882a571-29af-4008-949e-6a31e89248e5 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.740239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecb732-a1da-4ba3-9982-d5c96440b65c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:34.779483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114abf2-20d7-4ff6-b673-6ddb55dc262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:34.780307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114abf2-20d7-4ff6-b673-6ddb55dc262e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.780419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114abf2-20d7-4ff6-b673-6ddb55dc262e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.780623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114abf2-20d7-4ff6-b673-6ddb55dc262e None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.781492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114abf2-20d7-4ff6-b673-6ddb55dc262e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:34.781761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114abf2-20d7-4ff6-b673-6ddb55dc262e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:34.783087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114abf2-20d7-4ff6-b673-6ddb55dc262e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:34.796323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 340/658] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:34.799165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40894a-2c50-4192-9d1d-6f2f2c44520b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.799681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40894a-2c50-4192-9d1d-6f2f2c44520b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.800316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40894a-2c50-4192-9d1d-6f2f2c44520b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.801182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:34.819130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4d40894a-2c50-4192-9d1d-6f2f2c44520b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:34.821135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4d40894a-2c50-4192-9d1d-6f2f2c44520b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:34.821628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882a571-29af-4008-949e-6a31e89248e5 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:34.869187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d5ed6-b3c1-4df3-9cef-5b83abc7a61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:34.869846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:34.870789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d5ed6-b3c1-4df3-9cef-5b83abc7a61e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:34.871085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d5ed6-b3c1-4df3-9cef-5b83abc7a61e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:34.871560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d5ed6-b3c1-4df3-9cef-5b83abc7a61e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:34.880050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d5ed6-b3c1-4df3-9cef-5b83abc7a61e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:34.881551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d5ed6-b3c1-4df3-9cef-5b83abc7a61e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:34.901268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 341/659] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:34.902284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40894a-2c50-4192-9d1d-6f2f2c44520b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-ec5d5ed6-b3c1-4df3-9cef-5b83abc7a61e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:34.903237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:34.904298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:16:34.905436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:34.907850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 319/660] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:34.909718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d5ed6-b3c1-4df3-9cef-5b83abc7a61e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.018576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40894a-2c50-4192-9d1d-6f2f2c44520b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.025158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d5ed6-b3c1-4df3-9cef-5b83abc7a61e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.056522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f46483-bb2b-439e-b1c4-181937d77120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.057334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f46483-bb2b-439e-b1c4-181937d77120 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.057621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f46483-bb2b-439e-b1c4-181937d77120 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.057904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f46483-bb2b-439e-b1c4-181937d77120 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.063771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d630b-d7c3-466e-a4d5-9a22b61b4e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.064564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d630b-d7c3-466e-a4d5-9a22b61b4e8d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.064853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d630b-d7c3-466e-a4d5-9a22b61b4e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.065151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d630b-d7c3-466e-a4d5-9a22b61b4e8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.066333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f46483-bb2b-439e-b1c4-181937d77120 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:35.067933 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f46483-bb2b-439e-b1c4-181937d77120 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:35.073016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d630b-d7c3-466e-a4d5-9a22b61b4e8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:35.074467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d630b-d7c3-466e-a4d5-9a22b61b4e8d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-e4f46483-bb2b-439e-b1c4-181937d77120 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:35.084599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:35.085972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 342/661] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:35.087374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f46483-bb2b-439e-b1c4-181937d77120 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-379d630b-d7c3-466e-a4d5-9a22b61b4e8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:35.093203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:35.094077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:35.094941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:35.094941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:35.094941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:35.094941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:35.094941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:35.094941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:35.094941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:35.094941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:35.094941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 320/662] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:35.100384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d630b-d7c3-466e-a4d5-9a22b61b4e8d None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.100384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:35.123246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:35.200115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f46483-bb2b-439e-b1c4-181937d77120 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.204287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d630b-d7c3-466e-a4d5-9a22b61b4e8d None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.237046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356c9e1-71e6-4500-bbee-a0dc881787b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.237719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356c9e1-71e6-4500-bbee-a0dc881787b9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.238056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356c9e1-71e6-4500-bbee-a0dc881787b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.238334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356c9e1-71e6-4500-bbee-a0dc881787b9 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.239219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356c9e1-71e6-4500-bbee-a0dc881787b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:35.239424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84fd8c-d4a2-479a-a6d2-c9a020dba034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.239545 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356c9e1-71e6-4500-bbee-a0dc881787b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:35.240224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84fd8c-d4a2-479a-a6d2-c9a020dba034 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.240507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84fd8c-d4a2-479a-a6d2-c9a020dba034 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.240812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84fd8c-d4a2-479a-a6d2-c9a020dba034 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.241000 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356c9e1-71e6-4500-bbee-a0dc881787b9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:35.253019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84fd8c-d4a2-479a-a6d2-c9a020dba034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:35.264164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 343/663] 10.222.0.30 () {56 vars in 1139 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:35.264164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356c9e1-71e6-4500-bbee-a0dc881787b9 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-3a84fd8c-d4a2-479a-a6d2-c9a020dba034 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:35.264384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:35.265333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 321/664] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:35.266168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84fd8c-d4a2-479a-a6d2-c9a020dba034 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.336659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:35.368785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:35.380529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356c9e1-71e6-4500-bbee-a0dc881787b9 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.389293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84fd8c-d4a2-479a-a6d2-c9a020dba034 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.416913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843c2f7-4890-4ebd-bc85-864ebc66cd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.417567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c2f7-4890-4ebd-bc85-864ebc66cd42 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.417992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c2f7-4890-4ebd-bc85-864ebc66cd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.418252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c2f7-4890-4ebd-bc85-864ebc66cd42 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.419213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843c2f7-4890-4ebd-bc85-864ebc66cd42 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:35.419521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843c2f7-4890-4ebd-bc85-864ebc66cd42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:35.420967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843c2f7-4890-4ebd-bc85-864ebc66cd42 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:35.423739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd83210-f56d-422b-acbe-a0433174cfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.424444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd83210-f56d-422b-acbe-a0433174cfff None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.424733 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd83210-f56d-422b-acbe-a0433174cfff None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.425059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd83210-f56d-422b-acbe-a0433174cfff None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.425879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd83210-f56d-422b-acbe-a0433174cfff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:35.426211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd83210-f56d-422b-acbe-a0433174cfff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:35.427809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd83210-f56d-422b-acbe-a0433174cfff None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:35.444919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 322/665] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:35.444919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd83210-f56d-422b-acbe-a0433174cfff None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.445769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 344/666] 10.222.0.30 () {56 vars in 1137 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:35.446717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843c2f7-4890-4ebd-bc85-864ebc66cd42 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.545882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd83210-f56d-422b-acbe-a0433174cfff None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.554170 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843c2f7-4890-4ebd-bc85-864ebc66cd42 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.570189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:35.583431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6dd31-671f-4412-9881-fcc31ccc6f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.584080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6dd31-671f-4412-9881-fcc31ccc6f18 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.584361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6dd31-671f-4412-9881-fcc31ccc6f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.584629 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6dd31-671f-4412-9881-fcc31ccc6f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.595856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2f6ff-15c9-49a2-a4c2-c5d03ca448c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.596463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2f6ff-15c9-49a2-a4c2-c5d03ca448c9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.596747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2f6ff-15c9-49a2-a4c2-c5d03ca448c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.597056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2f6ff-15c9-49a2-a4c2-c5d03ca448c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.597807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2f6ff-15c9-49a2-a4c2-c5d03ca448c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:35.598115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2f6ff-15c9-49a2-a4c2-c5d03ca448c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:35.599549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2f6ff-15c9-49a2-a4c2-c5d03ca448c9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:35.614551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 345/667] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:35.614551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2f6ff-15c9-49a2-a4c2-c5d03ca448c9 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.632708 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:35.679380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6dd31-671f-4412-9881-fcc31ccc6f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:35.681774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6dd31-671f-4412-9881-fcc31ccc6f18 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:35.734936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2f6ff-15c9-49a2-a4c2-c5d03ca448c9 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.771665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc6afe-a3f7-432f-84e7-a785dc2a0195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.772473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc6afe-a3f7-432f-84e7-a785dc2a0195 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.772764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc6afe-a3f7-432f-84e7-a785dc2a0195 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.773084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc6afe-a3f7-432f-84e7-a785dc2a0195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.797457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 323/668] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:35.797457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6dd31-671f-4412-9881-fcc31ccc6f18 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.823161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:35.867759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc6afe-a3f7-432f-84e7-a785dc2a0195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:35.869145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc6afe-a3f7-432f-84e7-a785dc2a0195 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:35.884965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:35.909354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6dd31-671f-4412-9881-fcc31ccc6f18 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:35.945763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2afab1-e767-4d74-bd9a-709eecceeec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:35.946700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2afab1-e767-4d74-bd9a-709eecceeec8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:35.959579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2afab1-e767-4d74-bd9a-709eecceeec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:35.959894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2afab1-e767-4d74-bd9a-709eecceeec8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:35.960748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2afab1-e767-4d74-bd9a-709eecceeec8 None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=232cfaefaf584ac59720713e02cf2c2d) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:35.961093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2afab1-e767-4d74-bd9a-709eecceeec8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:35.962558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2afab1-e767-4d74-bd9a-709eecceeec8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:35.969424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 324/669] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/role_assignments?user.id=232cfaefaf584ac59720713e02cf2c2d&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:35.970731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2afab1-e767-4d74-bd9a-709eecceeec8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:35.997708 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 346/670] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:35.998917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc6afe-a3f7-432f-84e7-a785dc2a0195 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.080683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:36.099129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2afab1-e767-4d74-bd9a-709eecceeec8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.113309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc6afe-a3f7-432f-84e7-a785dc2a0195 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.129826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:36.141869 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02557cb2-ac10-4ae5-9cd2-6cec85930312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:36.143276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02557cb2-ac10-4ae5-9cd2-6cec85930312 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:36.143757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02557cb2-ac10-4ae5-9cd2-6cec85930312 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:36.144249 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02557cb2-ac10-4ae5-9cd2-6cec85930312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:36.155386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b9e9d-ef42-45ed-b650-5d8345d4cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:36.156138 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b9e9d-ef42-45ed-b650-5d8345d4cd2c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:36.156428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b9e9d-ef42-45ed-b650-5d8345d4cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:36.156707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b9e9d-ef42-45ed-b650-5d8345d4cd2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:36.158548 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02557cb2-ac10-4ae5-9cd2-6cec85930312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:36.160388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02557cb2-ac10-4ae5-9cd2-6cec85930312 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:36.169390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b9e9d-ef42-45ed-b650-5d8345d4cd2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:36.170912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b9e9d-ef42-45ed-b650-5d8345d4cd2c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-02557cb2-ac10-4ae5-9cd2-6cec85930312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:36.180507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:36.181401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:36.182261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:36.182261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:36.182261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:36.182261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:36.183824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 325/671] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:36.184759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02557cb2-ac10-4ae5-9cd2-6cec85930312 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-117b9e9d-ef42-45ed-b650-5d8345d4cd2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:36.190693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:36.191717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:36.192809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:36.192809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:36.192809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:36.192809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:36.193068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 347/672] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:36.193733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b9e9d-ef42-45ed-b650-5d8345d4cd2c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.325548 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02557cb2-ac10-4ae5-9cd2-6cec85930312 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.330701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b9e9d-ef42-45ed-b650-5d8345d4cd2c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.371817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd9bfc-3a71-49f2-81f2-3edef9ced355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:36.373145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd9bfc-3a71-49f2-81f2-3edef9ced355 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:36.373412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ee16e-b564-40f9-8f71-33b70ff8e005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:36.373734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd9bfc-3a71-49f2-81f2-3edef9ced355 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:36.374085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ee16e-b564-40f9-8f71-33b70ff8e005 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:36.374373 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ee16e-b564-40f9-8f71-33b70ff8e005 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:36.374480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd9bfc-3a71-49f2-81f2-3edef9ced355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:36.374651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ee16e-b564-40f9-8f71-33b70ff8e005 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:36.375552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ee16e-b564-40f9-8f71-33b70ff8e005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:36.375879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ee16e-b564-40f9-8f71-33b70ff8e005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:36.377586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ee16e-b564-40f9-8f71-33b70ff8e005 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:36.382938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:36.388260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd9bfc-3a71-49f2-81f2-3edef9ced355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:36.389914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd9bfc-3a71-49f2-81f2-3edef9ced355 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:36.395290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 348/673] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:36.396287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ee16e-b564-40f9-8f71-33b70ff8e005 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-72fd9bfc-3a71-49f2-81f2-3edef9ced355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:36.409201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:36.410089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:36.410949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:36.410949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:36.410949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:36.410949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:36.413133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 326/674] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:36.414251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd9bfc-3a71-49f2-81f2-3edef9ced355 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.427361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:36.524451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd9bfc-3a71-49f2-81f2-3edef9ced355 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.534701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ee16e-b564-40f9-8f71-33b70ff8e005 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.561896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b35714-6c9a-43c1-aa5a-47eedbf56a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:36.562795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b35714-6c9a-43c1-aa5a-47eedbf56a9b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:36.563080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b35714-6c9a-43c1-aa5a-47eedbf56a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:36.563354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b35714-6c9a-43c1-aa5a-47eedbf56a9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:36.572695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b35714-6c9a-43c1-aa5a-47eedbf56a9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:36.574314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b35714-6c9a-43c1-aa5a-47eedbf56a9b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:36.579257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e351bb3-2619-4641-84c7-16933854e962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:36.579929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e351bb3-2619-4641-84c7-16933854e962 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:36.580214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e351bb3-2619-4641-84c7-16933854e962 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:36.580485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e351bb3-2619-4641-84c7-16933854e962 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:36.581535 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e351bb3-2619-4641-84c7-16933854e962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:36.581841 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e351bb3-2619-4641-84c7-16933854e962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:36.583196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e351bb3-2619-4641-84c7-16933854e962 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b9b35714-6c9a-43c1-aa5a-47eedbf56a9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:36.589720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:36.590678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:36.591573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 327/675] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:36.593073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b35714-6c9a-43c1-aa5a-47eedbf56a9b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.605110 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 349/676] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:36.607913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e351bb3-2619-4641-84c7-16933854e962 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.661474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:36.707254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:36.725187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e351bb3-2619-4641-84c7-16933854e962 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.742362 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b35714-6c9a-43c1-aa5a-47eedbf56a9b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.768161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c016b-f565-4816-b34d-23c40567e9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:36.768845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c016b-f565-4816-b34d-23c40567e9fa None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:36.769176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c016b-f565-4816-b34d-23c40567e9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:36.769452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c016b-f565-4816-b34d-23c40567e9fa None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:36.770427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c016b-f565-4816-b34d-23c40567e9fa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:36.770604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c016b-f565-4816-b34d-23c40567e9fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:36.772063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c016b-f565-4816-b34d-23c40567e9fa None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:36.792589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 350/677] 10.222.0.30 () {56 vars in 1135 bytes} [Mon May 23 15:16:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:36.793385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19a9b1-ed2b-4713-8de9-08ef35bc4de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:36.793610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c016b-f565-4816-b34d-23c40567e9fa None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.794567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19a9b1-ed2b-4713-8de9-08ef35bc4de8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:36.795047 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19a9b1-ed2b-4713-8de9-08ef35bc4de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:36.795533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19a9b1-ed2b-4713-8de9-08ef35bc4de8 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:36.796830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19a9b1-ed2b-4713-8de9-08ef35bc4de8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:36.798500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19a9b1-ed2b-4713-8de9-08ef35bc4de8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:36.800118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19a9b1-ed2b-4713-8de9-08ef35bc4de8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:36.817199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 328/678] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:36.818571 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19a9b1-ed2b-4713-8de9-08ef35bc4de8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.900740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c016b-f565-4816-b34d-23c40567e9fa None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.938412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce117761-6b3d-41df-a73e-ce8ce476d01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:36.939253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce117761-6b3d-41df-a73e-ce8ce476d01e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:36.939538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce117761-6b3d-41df-a73e-ce8ce476d01e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:36.939815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce117761-6b3d-41df-a73e-ce8ce476d01e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:36.947423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce117761-6b3d-41df-a73e-ce8ce476d01e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:36.948858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce117761-6b3d-41df-a73e-ce8ce476d01e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:36.955356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19a9b1-ed2b-4713-8de9-08ef35bc4de8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-ce117761-6b3d-41df-a73e-ce8ce476d01e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:36.963286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:36.964576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:36.965645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:36.965645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:36.965645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:36.965645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:36.965645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:36.965645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:36.965645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:36.965645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:36.965645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 351/679] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:36.966131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce117761-6b3d-41df-a73e-ce8ce476d01e None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:36.968672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:36.972350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:37.013296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5cef6-f7b1-4f8e-8411-560237dafe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:37.014244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5cef6-f7b1-4f8e-8411-560237dafe1c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.014533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5cef6-f7b1-4f8e-8411-560237dafe1c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.014806 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5cef6-f7b1-4f8e-8411-560237dafe1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.107939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5cef6-f7b1-4f8e-8411-560237dafe1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:37.109450 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5cef6-f7b1-4f8e-8411-560237dafe1c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:37.109650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce117761-6b3d-41df-a73e-ce8ce476d01e None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:37.144728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcf8f6-f273-4859-a329-a997a0b4d5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:37.145553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcf8f6-f273-4859-a329-a997a0b4d5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.145839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcf8f6-f273-4859-a329-a997a0b4d5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.146114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcf8f6-f273-4859-a329-a997a0b4d5e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.153528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcf8f6-f273-4859-a329-a997a0b4d5e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:37.154957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcf8f6-f273-4859-a329-a997a0b4d5e1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-e2fcf8f6-f273-4859-a329-a997a0b4d5e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:37.169885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:37.170955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:37.171963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:37.171963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:37.171963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:37.171963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:37.171963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:37.171963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:37.171963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:37.171963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:37.171963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 352/680] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:37.172689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c06d09-ffac-43cf-b9c1-80bd6926e61b None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.173189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c06d09-ffac-43cf-b9c1-80bd6926e61b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.173305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c06d09-ffac-43cf-b9c1-80bd6926e61b None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.174629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 353/681] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:37.177928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c06d09-ffac-43cf-b9c1-80bd6926e61b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:37.223336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 329/682] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:37.224462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5cef6-f7b1-4f8e-8411-560237dafe1c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:37.238199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:37.248682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:37.292108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c06d09-ffac-43cf-b9c1-80bd6926e61b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:37.330882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddbbb8-ed70-4a49-afe3-241f46941dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:37.331836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddbbb8-ed70-4a49-afe3-241f46941dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.332260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddbbb8-ed70-4a49-afe3-241f46941dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.332704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddbbb8-ed70-4a49-afe3-241f46941dd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.333650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddbbb8-ed70-4a49-afe3-241f46941dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:37.334098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddbbb8-ed70-4a49-afe3-241f46941dd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:37.335698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddbbb8-ed70-4a49-afe3-241f46941dd4 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:37.340106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5cef6-f7b1-4f8e-8411-560237dafe1c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:37.348893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 354/683] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:37.350438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd421-3e38-4235-bae8-ed82392fae20 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.350853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd421-3e38-4235-bae8-ed82392fae20 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.351254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd421-3e38-4235-bae8-ed82392fae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.370512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-629fd421-3e38-4235-bae8-ed82392fae20 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:37.371759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-629fd421-3e38-4235-bae8-ed82392fae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:37.389672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43b270-963c-4f3b-aad7-220ee262dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:37.390337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43b270-963c-4f3b-aad7-220ee262dde9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.390616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43b270-963c-4f3b-aad7-220ee262dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.390894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43b270-963c-4f3b-aad7-220ee262dde9 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.391677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43b270-963c-4f3b-aad7-220ee262dde9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:37.391980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43b270-963c-4f3b-aad7-220ee262dde9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:37.393486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43b270-963c-4f3b-aad7-220ee262dde9 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:37.406468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 330/684] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:37.407330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43b270-963c-4f3b-aad7-220ee262dde9 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:37.450067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 355/685] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:37.450931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd421-3e38-4235-bae8-ed82392fae20 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:37.493814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:37.514065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:37.530293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43b270-963c-4f3b-aad7-220ee262dde9 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:37.563107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd421-3e38-4235-bae8-ed82392fae20 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:37.568114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd5b4-2ec5-49a3-a266-4e21ef5738a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:37.568933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd5b4-2ec5-49a3-a266-4e21ef5738a1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.569218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd5b4-2ec5-49a3-a266-4e21ef5738a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.569491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd5b4-2ec5-49a3-a266-4e21ef5738a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.581651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cd5b4-2ec5-49a3-a266-4e21ef5738a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:37.583273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cd5b4-2ec5-49a3-a266-4e21ef5738a1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:37.602009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3eebf-3f54-4eaf-83d9-5559ecae2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:37.602827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3eebf-3f54-4eaf-83d9-5559ecae2985 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b85cd5b4-2ec5-49a3-a266-4e21ef5738a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:37.603040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:37.603895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:37.605210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:37.605210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:37.605210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:37.605210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:37.608578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3eebf-3f54-4eaf-83d9-5559ecae2985 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.608578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3eebf-3f54-4eaf-83d9-5559ecae2985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.608578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3eebf-3f54-4eaf-83d9-5559ecae2985 None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=68d85e20dc884cd0bad5bb6c304776ca) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:37.609025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 331/686] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:37.609025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd5b4-2ec5-49a3-a266-4e21ef5738a1 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:37.609288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3eebf-3f54-4eaf-83d9-5559ecae2985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:37.610794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3eebf-3f54-4eaf-83d9-5559ecae2985 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:37.629934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 356/687] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:37 2022] GET /identity/v3/role_assignments?user.id=68d85e20dc884cd0bad5bb6c304776ca&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:37.629934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3eebf-3f54-4eaf-83d9-5559ecae2985 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:37.756251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd5b4-2ec5-49a3-a266-4e21ef5738a1 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:37.771918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3eebf-3f54-4eaf-83d9-5559ecae2985 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:37.786201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:37.794487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:37.801674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0055bca-e130-4a40-b571-d2130da9affb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:37.802451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0055bca-e130-4a40-b571-d2130da9affb None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.802758 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0055bca-e130-4a40-b571-d2130da9affb None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.803061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0055bca-e130-4a40-b571-d2130da9affb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.803995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0055bca-e130-4a40-b571-d2130da9affb None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=36daf80a4cbc4cfc8962c7924433d257) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:37.804334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0055bca-e130-4a40-b571-d2130da9affb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:37.806752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0055bca-e130-4a40-b571-d2130da9affb None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:37.820905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 332/688] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:37 2022] GET /identity/v3/role_assignments?user.id=36daf80a4cbc4cfc8962c7924433d257&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:37.820905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0055bca-e130-4a40-b571-d2130da9affb None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:37.829807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3be8a-e954-41a7-8a8f-7cce57d3cd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:37.830915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3be8a-e954-41a7-8a8f-7cce57d3cd07 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.831375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3be8a-e954-41a7-8a8f-7cce57d3cd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.831657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3be8a-e954-41a7-8a8f-7cce57d3cd07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.845529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3be8a-e954-41a7-8a8f-7cce57d3cd07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:37.847010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3be8a-e954-41a7-8a8f-7cce57d3cd07 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-b4b3be8a-e954-41a7-8a8f-7cce57d3cd07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:37.868278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:37.869268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:37.870159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:37.870159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:37.870159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:37.870159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:37.875549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 357/689] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:37.876536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3be8a-e954-41a7-8a8f-7cce57d3cd07 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:37.941224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0055bca-e130-4a40-b571-d2130da9affb None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:37.982057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6905d-dff2-4a58-a55a-483d9ef56afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:37.982835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6905d-dff2-4a58-a55a-483d9ef56afa None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:37.983071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6905d-dff2-4a58-a55a-483d9ef56afa None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:37.983350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6905d-dff2-4a58-a55a-483d9ef56afa None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:37.984169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6905d-dff2-4a58-a55a-483d9ef56afa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:37.984480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6905d-dff2-4a58-a55a-483d9ef56afa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:37.986007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6905d-dff2-4a58-a55a-483d9ef56afa None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:37.990615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3be8a-e954-41a7-8a8f-7cce57d3cd07 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:38.001307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 333/690] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:38.007424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6905d-dff2-4a58-a55a-483d9ef56afa None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:38.031263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed4bef-af2b-4deb-ac73-c87d7b3fae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:38.032053 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed4bef-af2b-4deb-ac73-c87d7b3fae43 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:38.032343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed4bef-af2b-4deb-ac73-c87d7b3fae43 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:38.032625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed4bef-af2b-4deb-ac73-c87d7b3fae43 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:38.033397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed4bef-af2b-4deb-ac73-c87d7b3fae43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:38.033708 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed4bef-af2b-4deb-ac73-c87d7b3fae43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:38.035162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed4bef-af2b-4deb-ac73-c87d7b3fae43 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:38.049055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 358/691] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:38.056064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed4bef-af2b-4deb-ac73-c87d7b3fae43 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:38.066444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:38.098877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:38.163886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6905d-dff2-4a58-a55a-483d9ef56afa None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:38.190827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed4bef-af2b-4deb-ac73-c87d7b3fae43 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:38.206195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a26d94-2857-4c3e-9fe2-48a3c25ce71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:38.207329 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a26d94-2857-4c3e-9fe2-48a3c25ce71b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:38.207812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a26d94-2857-4c3e-9fe2-48a3c25ce71b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:38.208308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a26d94-2857-4c3e-9fe2-48a3c25ce71b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:38.216303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a26d94-2857-4c3e-9fe2-48a3c25ce71b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:38.218140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a26d94-2857-4c3e-9fe2-48a3c25ce71b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:38.226306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f0019-38ab-434b-9df4-e239f04d213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:38.227506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f0019-38ab-434b-9df4-e239f04d213f None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:38.227790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f0019-38ab-434b-9df4-e239f04d213f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:38.228063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f0019-38ab-434b-9df4-e239f04d213f None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:38.228776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f0019-38ab-434b-9df4-e239f04d213f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:38.230256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f0019-38ab-434b-9df4-e239f04d213f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-98a26d94-2857-4c3e-9fe2-48a3c25ce71b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:38.235189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:38.236455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:38.237454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:38.237454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:38.237454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:38.237454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:38.237454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:38.237454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:38.237454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:38.237454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:38.240048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 334/692] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:38.246858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a26d94-2857-4c3e-9fe2-48a3c25ce71b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:38.255970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 359/693] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:16:38 2022] POST /identity/v3/users => generated 254 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:16:38.391281 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a26d94-2857-4c3e-9fe2-48a3c25ce71b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:38.401966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:38.439586 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acac7b-0606-42f0-962b-c7fae6dd6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:38.440537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acac7b-0606-42f0-962b-c7fae6dd6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:38.441043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acac7b-0606-42f0-962b-c7fae6dd6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:38.441484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acac7b-0606-42f0-962b-c7fae6dd6fb8 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:38.442423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acac7b-0606-42f0-962b-c7fae6dd6fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:38.442880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acac7b-0606-42f0-962b-c7fae6dd6fb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:38.444485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acac7b-0606-42f0-962b-c7fae6dd6fb8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:38.462846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 335/694] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:38.468506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f0019-38ab-434b-9df4-e239f04d213f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:38.512220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb14d-4859-4f4a-978f-8fcb36fcae99 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:38.512978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb14d-4859-4f4a-978f-8fcb36fcae99 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:38.513614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb14d-4859-4f4a-978f-8fcb36fcae99 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:38.515365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 336/695] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:38.522866 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38d6d-dbc2-4668-a47b-7e1e3e92c738 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:38.523197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38d6d-dbc2-4668-a47b-7e1e3e92c738 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:38.523726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38d6d-dbc2-4668-a47b-7e1e3e92c738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:38.556895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d8e38d6d-dbc2-4668-a47b-7e1e3e92c738 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:38.557119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d8e38d6d-dbc2-4668-a47b-7e1e3e92c738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:38.617958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:38.644744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f0019-38ab-434b-9df4-e239f04d213f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:38.676666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 337/696] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:38.686941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e8c9e-e573-4a35-b298-2ff54aa5a2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:38.687780 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e8c9e-e573-4a35-b298-2ff54aa5a2fe None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:38.688065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e8c9e-e573-4a35-b298-2ff54aa5a2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:38.688338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e8c9e-e573-4a35-b298-2ff54aa5a2fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:38.689346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e8c9e-e573-4a35-b298-2ff54aa5a2fe None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=ff6c489d950a4370bbf01866d2fc8fc3) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:38.689657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e8c9e-e573-4a35-b298-2ff54aa5a2fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:38.691076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e8c9e-e573-4a35-b298-2ff54aa5a2fe None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:38.698107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 360/697] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:38 2022] GET /identity/v3/role_assignments?user.id=ff6c489d950a4370bbf01866d2fc8fc3&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:38.709667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e38d6d-dbc2-4668-a47b-7e1e3e92c738 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:38.818686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:38.825271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e38d6d-dbc2-4668-a47b-7e1e3e92c738 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:38.863258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca13034-76be-4ad3-ac4b-11a5434bd766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:38.864072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca13034-76be-4ad3-ac4b-11a5434bd766 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:38.864351 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca13034-76be-4ad3-ac4b-11a5434bd766 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:38.864617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca13034-76be-4ad3-ac4b-11a5434bd766 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:38.876947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca13034-76be-4ad3-ac4b-11a5434bd766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-7ca13034-76be-4ad3-ac4b-11a5434bd766 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:38.884809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:38.885842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:38.886663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 338/698] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:38.892652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e8c9e-e573-4a35-b298-2ff54aa5a2fe None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:39.001353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e8c9e-e573-4a35-b298-2ff54aa5a2fe None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:39.038256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3789975-bfa0-4e32-bf80-c747dbe87b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:39.038924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3789975-bfa0-4e32-bf80-c747dbe87b71 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:39.039246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3789975-bfa0-4e32-bf80-c747dbe87b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:39.039539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3789975-bfa0-4e32-bf80-c747dbe87b71 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:39.040396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3789975-bfa0-4e32-bf80-c747dbe87b71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:39.040696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3789975-bfa0-4e32-bf80-c747dbe87b71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:39.042109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3789975-bfa0-4e32-bf80-c747dbe87b71 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:39.050922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 361/699] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:39.056369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca13034-76be-4ad3-ac4b-11a5434bd766 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:39.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca13034-76be-4ad3-ac4b-11a5434bd766 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:39.200106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b8bb8-a685-4db9-8a9d-c6badda0a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:39.200913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b8bb8-a685-4db9-8a9d-c6badda0a911 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:39.201197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b8bb8-a685-4db9-8a9d-c6badda0a911 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:39.201471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b8bb8-a685-4db9-8a9d-c6badda0a911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:39.213446 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b8bb8-a685-4db9-8a9d-c6badda0a911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:39.214922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b8bb8-a685-4db9-8a9d-c6badda0a911 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-342b8bb8-a685-4db9-8a9d-c6badda0a911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:39.232695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:39.234796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:39.236953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:39.236953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:39.236953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:39.236953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:39.236953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 339/700] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:39.240535 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3789975-bfa0-4e32-bf80-c747dbe87b71 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:39.254371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:39.357160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3789975-bfa0-4e32-bf80-c747dbe87b71 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:39.410583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f3e8b-22d2-43d7-96a2-6065df64d840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:39.411258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f3e8b-22d2-43d7-96a2-6065df64d840 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:39.411536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f3e8b-22d2-43d7-96a2-6065df64d840 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:39.411806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f3e8b-22d2-43d7-96a2-6065df64d840 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:39.412526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f3e8b-22d2-43d7-96a2-6065df64d840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:39.412827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f3e8b-22d2-43d7-96a2-6065df64d840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:39.414485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f3e8b-22d2-43d7-96a2-6065df64d840 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:39.572434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 362/701] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:39.577618 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b8bb8-a685-4db9-8a9d-c6badda0a911 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:39.605239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:39.690846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b8bb8-a685-4db9-8a9d-c6badda0a911 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:39.726524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a627b-435a-4ea1-862f-05e379871970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:39.727457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a627b-435a-4ea1-862f-05e379871970 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:39.727737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a627b-435a-4ea1-862f-05e379871970 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:39.728011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a627b-435a-4ea1-862f-05e379871970 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:39.735773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a627b-435a-4ea1-862f-05e379871970 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:39.737239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a627b-435a-4ea1-862f-05e379871970 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b54a627b-435a-4ea1-862f-05e379871970 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:39.751432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:39.752360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:39.753323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 340/702] 10.222.0.30 () {56 vars in 1093 bytes} [Mon May 23 15:16:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:39.762023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f3e8b-22d2-43d7-96a2-6065df64d840 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:39.858740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:39.893001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f3e8b-22d2-43d7-96a2-6065df64d840 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:39.937596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d6df2-ce09-46b1-bfc8-32e42bd1ad61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:39.938638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d6df2-ce09-46b1-bfc8-32e42bd1ad61 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:39.939124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d6df2-ce09-46b1-bfc8-32e42bd1ad61 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:39.939622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d6df2-ce09-46b1-bfc8-32e42bd1ad61 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:39.940660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d6df2-ce09-46b1-bfc8-32e42bd1ad61 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:39.941205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d6df2-ce09-46b1-bfc8-32e42bd1ad61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:39.942817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d6df2-ce09-46b1-bfc8-32e42bd1ad61 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:39.965196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 363/703] 10.222.0.30 () {56 vars in 1133 bytes} [Mon May 23 15:16:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:39.967910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a627b-435a-4ea1-862f-05e379871970 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:40.100657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:40.155615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a627b-435a-4ea1-862f-05e379871970 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:40.212539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b3700-0a25-4dd1-b4d9-5c8b9c58ffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:40.213221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b3700-0a25-4dd1-b4d9-5c8b9c58ffa3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:40.213501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b3700-0a25-4dd1-b4d9-5c8b9c58ffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:40.213776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b3700-0a25-4dd1-b4d9-5c8b9c58ffa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:40.316517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b3700-0a25-4dd1-b4d9-5c8b9c58ffa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:40.318060 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b3700-0a25-4dd1-b4d9-5c8b9c58ffa3 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:40.342327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6478c3-f544-4ba9-ae07-9a702ec6990b None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:40.342830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6478c3-f544-4ba9-ae07-9a702ec6990b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:40.342937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6478c3-f544-4ba9-ae07-9a702ec6990b None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:40.344518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 364/704] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:40.349542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3db768-5e7e-448d-93c1-676e319e7aed None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:40.349828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3db768-5e7e-448d-93c1-676e319e7aed None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:40.350107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3db768-5e7e-448d-93c1-676e319e7aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:40.368087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-2f3db768-5e7e-448d-93c1-676e319e7aed None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:40.369416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2f3db768-5e7e-448d-93c1-676e319e7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:40.421348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:40.446626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 341/705] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:40.450788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b3700-0a25-4dd1-b4d9-5c8b9c58ffa3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:40.519873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 365/706] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:40.558231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3db768-5e7e-448d-93c1-676e319e7aed None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:40.596519 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b3700-0a25-4dd1-b4d9-5c8b9c58ffa3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:40.638086 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d362c7e-2fe8-4293-88b9-9f8a2a17a641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:40.638931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d362c7e-2fe8-4293-88b9-9f8a2a17a641 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:40.639142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d362c7e-2fe8-4293-88b9-9f8a2a17a641 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:40.639459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d362c7e-2fe8-4293-88b9-9f8a2a17a641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:40.650276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d362c7e-2fe8-4293-88b9-9f8a2a17a641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:40.651803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d362c7e-2fe8-4293-88b9-9f8a2a17a641 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:40.657880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-1d362c7e-2fe8-4293-88b9-9f8a2a17a641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:40.672484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:40.673620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:40.674614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:40.674614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:40.674614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:40.674614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:40.674614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 342/707] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:40.675306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6aff6-b4c1-4660-999f-d20eb34f5d4e None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:40.675602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6aff6-b4c1-4660-999f-d20eb34f5d4e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:40.675852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6aff6-b4c1-4660-999f-d20eb34f5d4e None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:40.677623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 343/708] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:40.679339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6aff6-b4c1-4660-999f-d20eb34f5d4e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:40.703908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:40.724469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3db768-5e7e-448d-93c1-676e319e7aed None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:40.767738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a5786-853f-47ff-a4b4-861f4e900d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:40.769475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a5786-853f-47ff-a4b4-861f4e900d68 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:40.769764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a5786-853f-47ff-a4b4-861f4e900d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:40.770044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a5786-853f-47ff-a4b4-861f4e900d68 None admin] PATH_INFO: `/v3/services/network` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:40.770715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a5786-853f-47ff-a4b4-861f4e900d68 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:40.772126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a5786-853f-47ff-a4b4-861f4e900d68 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-ee5a5786-853f-47ff-a4b4-861f4e900d68 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:40.792577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:40.793652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:16:40.794500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:40.797511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 366/709] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:40 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:40.799143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06304e7f-61cd-4395-8cd2-be060eeb068d None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:40.799741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06304e7f-61cd-4395-8cd2-be060eeb068d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:40.800224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06304e7f-61cd-4395-8cd2-be060eeb068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:40.807361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6aff6-b4c1-4660-999f-d20eb34f5d4e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:40.819273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-06304e7f-61cd-4395-8cd2-be060eeb068d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:40.821698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-06304e7f-61cd-4395-8cd2-be060eeb068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:40.852399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42be05-550b-410a-871c-eab4080c58b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:40.853162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42be05-550b-410a-871c-eab4080c58b6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:40.853450 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42be05-550b-410a-871c-eab4080c58b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:40.853724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42be05-550b-410a-871c-eab4080c58b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:40.854717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42be05-550b-410a-871c-eab4080c58b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:40.855063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42be05-550b-410a-871c-eab4080c58b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:40.857137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42be05-550b-410a-871c-eab4080c58b6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:40.872598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 344/710] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:40.873762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42be05-550b-410a-871c-eab4080c58b6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:40.898271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 367/711] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:40.899748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06304e7f-61cd-4395-8cd2-be060eeb068d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:40.963382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:40.986065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:41.012153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42be05-550b-410a-871c-eab4080c58b6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:41.019867 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06304e7f-61cd-4395-8cd2-be060eeb068d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:41.057075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5744-8192-4691-84ed-34fa21fd8a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:41.057754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5744-8192-4691-84ed-34fa21fd8a54 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.058032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5744-8192-4691-84ed-34fa21fd8a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.058312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5744-8192-4691-84ed-34fa21fd8a54 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.059033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5744-8192-4691-84ed-34fa21fd8a54 None admin] RBAC: Adding query filter params (name=network) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:41.059343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5744-8192-4691-84ed-34fa21fd8a54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:41.059917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f9cdd-d6e3-4090-8150-28f94e843f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:41.060738 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5744-8192-4691-84ed-34fa21fd8a54 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:41.061185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f9cdd-d6e3-4090-8150-28f94e843f8a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.061657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f9cdd-d6e3-4090-8150-28f94e843f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.062215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f9cdd-d6e3-4090-8150-28f94e843f8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.066648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 345/712] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:16:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:41.068029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5744-8192-4691-84ed-34fa21fd8a54 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:41.072018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f9cdd-d6e3-4090-8150-28f94e843f8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:41.074041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f9cdd-d6e3-4090-8150-28f94e843f8a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-c91f9cdd-d6e3-4090-8150-28f94e843f8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:41.090485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:41.091454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:41.092384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:41.092384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:41.092384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:41.092384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:41.092384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:41.092384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:41.092384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:41.092384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:41.094993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 368/713] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:41.096160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f9cdd-d6e3-4090-8150-28f94e843f8a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:41.199822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5744-8192-4691-84ed-34fa21fd8a54 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:41.216524 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f9cdd-d6e3-4090-8150-28f94e843f8a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:41.233742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:41.239572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12df329-10cb-4aaa-89f8-969efa20d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:41.240719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12df329-10cb-4aaa-89f8-969efa20d855 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.241231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12df329-10cb-4aaa-89f8-969efa20d855 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.241699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12df329-10cb-4aaa-89f8-969efa20d855 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.243609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12df329-10cb-4aaa-89f8-969efa20d855 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:41.245750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12df329-10cb-4aaa-89f8-969efa20d855 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:41.262721 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bea2c-d284-46ac-8be8-a084a15c1925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:41.263678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bea2c-d284-46ac-8be8-a084a15c1925 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.264155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bea2c-d284-46ac-8be8-a084a15c1925 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.264640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bea2c-d284-46ac-8be8-a084a15c1925 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.265624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bea2c-d284-46ac-8be8-a084a15c1925 None admin] RBAC: Adding query filter params (type=network) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b12df329-10cb-4aaa-89f8-969efa20d855 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:41.266083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:41.266999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:41.267820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 346/714] 10.222.0.30 () {56 vars in 1129 bytes} [Mon May 23 15:16:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:41.268641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12df329-10cb-4aaa-89f8-969efa20d855 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:41.269415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bea2c-d284-46ac-8be8-a084a15c1925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:41.271051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bea2c-d284-46ac-8be8-a084a15c1925 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:41.278703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 369/715] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:16:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:41.279809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bea2c-d284-46ac-8be8-a084a15c1925 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:41.285077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:41.400670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bea2c-d284-46ac-8be8-a084a15c1925 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:41.444072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f51f4-fdc5-484e-b218-0de1b6e0b0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:41.445270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f51f4-fdc5-484e-b218-0de1b6e0b0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.446709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f51f4-fdc5-484e-b218-0de1b6e0b0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.447211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f51f4-fdc5-484e-b218-0de1b6e0b0e1 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.448131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f51f4-fdc5-484e-b218-0de1b6e0b0e1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:41.448651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f51f4-fdc5-484e-b218-0de1b6e0b0e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:41.450245 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f51f4-fdc5-484e-b218-0de1b6e0b0e1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:41.456004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 370/716] 10.222.0.30 () {56 vars in 1124 bytes} [Mon May 23 15:16:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:41.462678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f51f4-fdc5-484e-b218-0de1b6e0b0e1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:41.516603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:41.541086 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12df329-10cb-4aaa-89f8-969efa20d855 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:41.582541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f51f4-fdc5-484e-b218-0de1b6e0b0e1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:41.616838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f581f-f0ac-4578-969f-72ed1121f0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:41.618095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f581f-f0ac-4578-969f-72ed1121f0be None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.618390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f581f-f0ac-4578-969f-72ed1121f0be None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.618666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f581f-f0ac-4578-969f-72ed1121f0be None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.619877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f581f-f0ac-4578-969f-72ed1121f0be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:41.620196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f581f-f0ac-4578-969f-72ed1121f0be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:41.621965 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f581f-f0ac-4578-969f-72ed1121f0be None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:41.628427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc96f8-a705-41ba-9d84-64b5007cf861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:41.630186 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc96f8-a705-41ba-9d84-64b5007cf861 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.630679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc96f8-a705-41ba-9d84-64b5007cf861 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.631228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc96f8-a705-41ba-9d84-64b5007cf861 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.632179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc96f8-a705-41ba-9d84-64b5007cf861 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:41.632683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc96f8-a705-41ba-9d84-64b5007cf861 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:41.634929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc96f8-a705-41ba-9d84-64b5007cf861 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:41.641486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 371/717] 10.222.0.30 () {56 vars in 1124 bytes} [Mon May 23 15:16:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:41.644699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 347/718] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:41.648116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1dbf1-0e62-4b92-9838-103b32e2d657 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.648770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1dbf1-0e62-4b92-9838-103b32e2d657 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.649338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1dbf1-0e62-4b92-9838-103b32e2d657 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.651361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f581f-f0ac-4578-969f-72ed1121f0be None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:41.652715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 372/719] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:41.658347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337f7bf-e649-491a-b3d2-69265ef4bdd0 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.658642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337f7bf-e649-491a-b3d2-69265ef4bdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.658897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337f7bf-e649-491a-b3d2-69265ef4bdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.682070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-5337f7bf-e649-491a-b3d2-69265ef4bdd0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:41.683290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5337f7bf-e649-491a-b3d2-69265ef4bdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:41.791312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:41.826911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 373/720] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:41.848100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f581f-f0ac-4578-969f-72ed1121f0be None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:41.862553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337f7bf-e649-491a-b3d2-69265ef4bdd0 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:41.898414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8efd86-f669-486e-be39-5c9ed555e7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:41.900113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8efd86-f669-486e-be39-5c9ed555e7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.900598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8efd86-f669-486e-be39-5c9ed555e7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.901106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8efd86-f669-486e-be39-5c9ed555e7f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.902251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8efd86-f669-486e-be39-5c9ed555e7f3 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=232cfaefaf584ac59720713e02cf2c2d) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:41.902764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8efd86-f669-486e-be39-5c9ed555e7f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:41.904504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8efd86-f669-486e-be39-5c9ed555e7f3 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:41.906333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:41.915967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 348/721] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:41 2022] GET /identity/v3/role_assignments?user.id=232cfaefaf584ac59720713e02cf2c2d&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:41.921872 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d8ea1-0954-4ae2-b8c3-d7058cdc801e None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.922166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d8ea1-0954-4ae2-b8c3-d7058cdc801e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.922428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d8ea1-0954-4ae2-b8c3-d7058cdc801e None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.924110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 349/722] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:41.930307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb75c4-588f-402e-8cf6-3460ffea58f3 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:41.930613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb75c4-588f-402e-8cf6-3460ffea58f3 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:41.930846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb75c4-588f-402e-8cf6-3460ffea58f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:41.950272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-aefb75c4-588f-402e-8cf6-3460ffea58f3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:41.951491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-aefb75c4-588f-402e-8cf6-3460ffea58f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:42.017477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337f7bf-e649-491a-b3d2-69265ef4bdd0 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:42.093170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 350/723] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:16:42.101752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b1540-5e65-4072-a318-4ca5d67266d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:42.103016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b1540-5e65-4072-a318-4ca5d67266d7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:42.103308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b1540-5e65-4072-a318-4ca5d67266d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:42.103589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b1540-5e65-4072-a318-4ca5d67266d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:42.118058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b1540-5e65-4072-a318-4ca5d67266d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-438b1540-5e65-4072-a318-4ca5d67266d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:42.126558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:42.127500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:42.128753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 374/724] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:42.134722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb75c4-588f-402e-8cf6-3460ffea58f3 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:42.136094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b1540-5e65-4072-a318-4ca5d67266d7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:42.210226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:42.256345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb75c4-588f-402e-8cf6-3460ffea58f3 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:42.298950 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:42.309942 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b1540-5e65-4072-a318-4ca5d67266d7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:42.331337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e33c6-465a-4e74-b7b6-5dbf4becbb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:42.332203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e33c6-465a-4e74-b7b6-5dbf4becbb4b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:42.332538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e33c6-465a-4e74-b7b6-5dbf4becbb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:42.332979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e33c6-465a-4e74-b7b6-5dbf4becbb4b None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:42.333792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e33c6-465a-4e74-b7b6-5dbf4becbb4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:42.334125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e33c6-465a-4e74-b7b6-5dbf4becbb4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:42.335694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e33c6-465a-4e74-b7b6-5dbf4becbb4b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:42.355539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 351/725] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:42.360738 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e33c6-465a-4e74-b7b6-5dbf4becbb4b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:42.364033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bd640-bafc-4a89-9af4-ab131bd9b479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:42.365002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bd640-bafc-4a89-9af4-ab131bd9b479 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:42.365293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bd640-bafc-4a89-9af4-ab131bd9b479 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:42.365570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bd640-bafc-4a89-9af4-ab131bd9b479 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:42.384792 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bd640-bafc-4a89-9af4-ab131bd9b479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-670bd640-bafc-4a89-9af4-ab131bd9b479 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:42.393374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:42.394382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:42.395682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 375/726] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:42.406002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bd640-bafc-4a89-9af4-ab131bd9b479 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:42.473322 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e33c6-465a-4e74-b7b6-5dbf4becbb4b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:42.500483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:42.513668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445a83b-0d4d-473f-bd88-985a9553a4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:42.514628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445a83b-0d4d-473f-bd88-985a9553a4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:42.514929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445a83b-0d4d-473f-bd88-985a9553a4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:42.515210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445a83b-0d4d-473f-bd88-985a9553a4c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:42.527695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bd640-bafc-4a89-9af4-ab131bd9b479 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:42.528096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445a83b-0d4d-473f-bd88-985a9553a4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:42.529808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445a83b-0d4d-473f-bd88-985a9553a4c5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-a445a83b-0d4d-473f-bd88-985a9553a4c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:42.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:42.552264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:42.554569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:42.554569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:42.554569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:42.554569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:42.554569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 352/727] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:42.559484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445a83b-0d4d-473f-bd88-985a9553a4c5 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:42.566596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d19130-6afc-4d1b-a83c-4a9b8b5bdcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:42.567270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d19130-6afc-4d1b-a83c-4a9b8b5bdcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:42.567559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d19130-6afc-4d1b-a83c-4a9b8b5bdcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:42.567837 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d19130-6afc-4d1b-a83c-4a9b8b5bdcd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:42.568586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d19130-6afc-4d1b-a83c-4a9b8b5bdcd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:42.568909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d19130-6afc-4d1b-a83c-4a9b8b5bdcd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:42.570271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d19130-6afc-4d1b-a83c-4a9b8b5bdcd7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:42.590152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 376/728] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:42.599967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d19130-6afc-4d1b-a83c-4a9b8b5bdcd7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:42.636635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:42.746900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d19130-6afc-4d1b-a83c-4a9b8b5bdcd7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:42.781148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445a83b-0d4d-473f-bd88-985a9553a4c5 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:42.800709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec9494-06a3-431a-939c-c7a099924249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:42.801890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec9494-06a3-431a-939c-c7a099924249 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:42.802522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec9494-06a3-431a-939c-c7a099924249 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:42.804108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec9494-06a3-431a-939c-c7a099924249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:42.819606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec9494-06a3-431a-939c-c7a099924249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:42.821243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec9494-06a3-431a-939c-c7a099924249 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-93ec9494-06a3-431a-939c-c7a099924249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:42.851579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:42.853155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:42.855419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:42.855419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:42.855419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:42.855419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:42.855419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 377/729] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:42.858793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969714f-26cc-49d7-91d4-b929d01a5625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:42.859509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969714f-26cc-49d7-91d4-b929d01a5625 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:42.859848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969714f-26cc-49d7-91d4-b929d01a5625 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:42.860093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969714f-26cc-49d7-91d4-b929d01a5625 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:42.860861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969714f-26cc-49d7-91d4-b929d01a5625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:42.861227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969714f-26cc-49d7-91d4-b929d01a5625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:42.862661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969714f-26cc-49d7-91d4-b929d01a5625 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:42.865024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec9494-06a3-431a-939c-c7a099924249 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:42.886396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 353/730] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:42.891433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969714f-26cc-49d7-91d4-b929d01a5625 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:42.955076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:42.994511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:43.055983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969714f-26cc-49d7-91d4-b929d01a5625 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:43.059247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec9494-06a3-431a-939c-c7a099924249 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:43.115731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e47ab5-ece4-49de-be39-57bdbc1d3cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:43.116327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e47ab5-ece4-49de-be39-57bdbc1d3cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:43.116613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e47ab5-ece4-49de-be39-57bdbc1d3cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:43.116910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e47ab5-ece4-49de-be39-57bdbc1d3cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:43.117878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e47ab5-ece4-49de-be39-57bdbc1d3cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:43.118202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e47ab5-ece4-49de-be39-57bdbc1d3cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:43.119569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e47ab5-ece4-49de-be39-57bdbc1d3cb2 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:43.122650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0cb31-a072-438e-8047-9300f160903b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:43.123458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0cb31-a072-438e-8047-9300f160903b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:43.123738 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0cb31-a072-438e-8047-9300f160903b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:43.124011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0cb31-a072-438e-8047-9300f160903b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:43.134121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0cb31-a072-438e-8047-9300f160903b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:43.135603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0cb31-a072-438e-8047-9300f160903b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:43.151894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 378/731] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:43.152812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e47ab5-ece4-49de-be39-57bdbc1d3cb2 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-f5f0cb31-a072-438e-8047-9300f160903b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:43.162576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:43.163491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:16:43.165076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:43.176743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 354/732] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:16:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:43.177759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0cb31-a072-438e-8047-9300f160903b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:43.323778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e47ab5-ece4-49de-be39-57bdbc1d3cb2 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:43.354024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:43.362572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0cb31-a072-438e-8047-9300f160903b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:43.391268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:43.402166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18317c62-a01b-4164-a0ae-3dfcc1ed3bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:43.403150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18317c62-a01b-4164-a0ae-3dfcc1ed3bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:43.403643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18317c62-a01b-4164-a0ae-3dfcc1ed3bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:43.404125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18317c62-a01b-4164-a0ae-3dfcc1ed3bd1 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:43.405236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18317c62-a01b-4164-a0ae-3dfcc1ed3bd1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:43.405772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18317c62-a01b-4164-a0ae-3dfcc1ed3bd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:43.407565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18317c62-a01b-4164-a0ae-3dfcc1ed3bd1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:43.417684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b76476-3ef0-456b-a6f9-2896fcc86e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:43.419343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b76476-3ef0-456b-a6f9-2896fcc86e9c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:43.419343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b76476-3ef0-456b-a6f9-2896fcc86e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:43.419343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b76476-3ef0-456b-a6f9-2896fcc86e9c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:43.428822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 355/733] 10.222.0.30 () {56 vars in 1149 bytes} [Mon May 23 15:16:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:43.430638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b76476-3ef0-456b-a6f9-2896fcc86e9c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:43.432306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b76476-3ef0-456b-a6f9-2896fcc86e9c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:43.434194 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18317c62-a01b-4164-a0ae-3dfcc1ed3bd1 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-e2b76476-3ef0-456b-a6f9-2896fcc86e9c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:43.467231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:43.468322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:43.469309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:43.478036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 379/734] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:43.480419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b76476-3ef0-456b-a6f9-2896fcc86e9c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:43.577808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18317c62-a01b-4164-a0ae-3dfcc1ed3bd1 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:43.618300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abe936-5fe6-42bf-ad35-14e7b9f48063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:43.620565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abe936-5fe6-42bf-ad35-14e7b9f48063 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:43.622204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abe936-5fe6-42bf-ad35-14e7b9f48063 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:43.622960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abe936-5fe6-42bf-ad35-14e7b9f48063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:43.682545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:43.696771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b76476-3ef0-456b-a6f9-2896fcc86e9c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:43.765696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1399a-0b46-4d8d-95d2-f93eebaf2f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:43.766379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1399a-0b46-4d8d-95d2-f93eebaf2f6d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:43.766662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1399a-0b46-4d8d-95d2-f93eebaf2f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:43.766935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1399a-0b46-4d8d-95d2-f93eebaf2f6d None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:43.767782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1399a-0b46-4d8d-95d2-f93eebaf2f6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:43.768095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1399a-0b46-4d8d-95d2-f93eebaf2f6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:43.769552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1399a-0b46-4d8d-95d2-f93eebaf2f6d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:43.775700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abe936-5fe6-42bf-ad35-14e7b9f48063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:43.777210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abe936-5fe6-42bf-ad35-14e7b9f48063 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:43.794117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 380/735] 10.222.0.30 () {56 vars in 1135 bytes} [Mon May 23 15:16:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:43.800637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1399a-0b46-4d8d-95d2-f93eebaf2f6d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:43.888831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:43.965653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 356/736] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:16:43.972235 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abe936-5fe6-42bf-ad35-14e7b9f48063 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:43.987146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1399a-0b46-4d8d-95d2-f93eebaf2f6d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:44.044217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95e460-6ddc-44bd-b0ca-3a9cf5d5a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:44.044909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95e460-6ddc-44bd-b0ca-3a9cf5d5a673 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.045192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95e460-6ddc-44bd-b0ca-3a9cf5d5a673 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.045467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95e460-6ddc-44bd-b0ca-3a9cf5d5a673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.053939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:44.105206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abe936-5fe6-42bf-ad35-14e7b9f48063 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:44.140169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95e460-6ddc-44bd-b0ca-3a9cf5d5a673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:44.142475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95e460-6ddc-44bd-b0ca-3a9cf5d5a673 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:44.144621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a938047-6021-46ec-aee7-35f5c7b4aef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:44.145447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a938047-6021-46ec-aee7-35f5c7b4aef5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.145731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a938047-6021-46ec-aee7-35f5c7b4aef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.146007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a938047-6021-46ec-aee7-35f5c7b4aef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.158913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a938047-6021-46ec-aee7-35f5c7b4aef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:44.160377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a938047-6021-46ec-aee7-35f5c7b4aef5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-0a938047-6021-46ec-aee7-35f5c7b4aef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:44.179094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:44.180015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:44.182238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:44.182238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:44.182238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:44.182238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:44.182238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 357/737] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:44.182542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd54be-f05b-48e9-abcc-61798d8ef703 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.182997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd54be-f05b-48e9-abcc-61798d8ef703 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.183090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd54be-f05b-48e9-abcc-61798d8ef703 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.185058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 358/738] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:44.185058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd54be-f05b-48e9-abcc-61798d8ef703 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:44.208349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:44.279354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 381/739] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:16:44.280709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c51b2-f212-44ce-a454-808e3e686390 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.281026 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c51b2-f212-44ce-a454-808e3e686390 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.281283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c51b2-f212-44ce-a454-808e3e686390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.298875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a25c51b2-f212-44ce-a454-808e3e686390 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:44.300103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a25c51b2-f212-44ce-a454-808e3e686390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:44.333231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd54be-f05b-48e9-abcc-61798d8ef703 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:44.342228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:44.372753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c1a34-cac4-4c1b-a1b5-54318c1dd76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:44.373533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c1a34-cac4-4c1b-a1b5-54318c1dd76a None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.373812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c1a34-cac4-4c1b-a1b5-54318c1dd76a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.374017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c1a34-cac4-4c1b-a1b5-54318c1dd76a None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.374753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c1a34-cac4-4c1b-a1b5-54318c1dd76a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:44.375062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c1a34-cac4-4c1b-a1b5-54318c1dd76a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:44.376441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c1a34-cac4-4c1b-a1b5-54318c1dd76a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:44.389723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 359/740] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:44.390808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c1a34-cac4-4c1b-a1b5-54318c1dd76a None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:44.404602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 382/741] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:44.404602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c51b2-f212-44ce-a454-808e3e686390 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:44.483324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:44.507959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c51b2-f212-44ce-a454-808e3e686390 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:44.518373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c1a34-cac4-4c1b-a1b5-54318c1dd76a None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:44.542408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1168d4-1f48-406a-be11-98c796c34036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:44.543135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1168d4-1f48-406a-be11-98c796c34036 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.543418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1168d4-1f48-406a-be11-98c796c34036 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.543696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1168d4-1f48-406a-be11-98c796c34036 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.550255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1168d4-1f48-406a-be11-98c796c34036 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:44.551692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1168d4-1f48-406a-be11-98c796c34036 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:44.551943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cff703-d1b0-4c00-bbbd-9ff9d62193b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:44.552723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cff703-d1b0-4c00-bbbd-9ff9d62193b0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.553039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cff703-d1b0-4c00-bbbd-9ff9d62193b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.553314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cff703-d1b0-4c00-bbbd-9ff9d62193b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.564123 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cff703-d1b0-4c00-bbbd-9ff9d62193b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:44.565574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cff703-d1b0-4c00-bbbd-9ff9d62193b0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-bb1168d4-1f48-406a-be11-98c796c34036 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:44.565900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:44.566968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:44.567866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:44.567866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:44.567866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:44.567866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:44.567866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:44.567866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:44.567866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:44.567866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:44.567866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 383/742] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:44.568358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be97c8a-4c2a-4831-9c87-75452050663d None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.568554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be97c8a-4c2a-4831-9c87-75452050663d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.568852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be97c8a-4c2a-4831-9c87-75452050663d None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.570287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 384/743] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:44.571467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be97c8a-4c2a-4831-9c87-75452050663d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b6cff703-d1b0-4c00-bbbd-9ff9d62193b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:44.585414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:44.586735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:44.587625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:44.587625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:44.587625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:44.587625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:44.587625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 360/744] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:44.588755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcffcc2-b446-4e8e-8896-4ac055ceff1f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.589299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcffcc2-b446-4e8e-8896-4ac055ceff1f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.589784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcffcc2-b446-4e8e-8896-4ac055ceff1f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.591323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 361/745] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:44.592045 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcffcc2-b446-4e8e-8896-4ac055ceff1f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:44.615085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:44.679841 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be97c8a-4c2a-4831-9c87-75452050663d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:44.697229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcffcc2-b446-4e8e-8896-4ac055ceff1f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:44.713065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b4684-5f8d-4048-ad83-bafedf7d6110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:44.713742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b4684-5f8d-4048-ad83-bafedf7d6110 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.714024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b4684-5f8d-4048-ad83-bafedf7d6110 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.714306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b4684-5f8d-4048-ad83-bafedf7d6110 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.715041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b4684-5f8d-4048-ad83-bafedf7d6110 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:44.716415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b4684-5f8d-4048-ad83-bafedf7d6110 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:44.720367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:44.733092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35c87d-8ef5-4abc-85fd-c707918f677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:44.733755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c87d-8ef5-4abc-85fd-c707918f677b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.734038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c87d-8ef5-4abc-85fd-c707918f677b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.734316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c87d-8ef5-4abc-85fd-c707918f677b None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.735108 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35c87d-8ef5-4abc-85fd-c707918f677b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:44.735422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35c87d-8ef5-4abc-85fd-c707918f677b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:44.736906 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35c87d-8ef5-4abc-85fd-c707918f677b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:44.737046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 385/746] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:16:44 2022] POST /identity/v3/services => generated 240 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:16:44.738485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e85ce-a9a1-4de5-a983-d56217f36b04 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.739016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e85ce-a9a1-4de5-a983-d56217f36b04 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.739352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e85ce-a9a1-4de5-a983-d56217f36b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.752233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 362/747] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:44.753153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35c87d-8ef5-4abc-85fd-c707918f677b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:44.763045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-fd1e85ce-a9a1-4de5-a983-d56217f36b04 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:44.764296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fd1e85ce-a9a1-4de5-a983-d56217f36b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:44.850309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:44.853497 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 386/748] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:44 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:44.866436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972a954-ecb7-4670-8767-bf5f28641fa2 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.866738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972a954-ecb7-4670-8767-bf5f28641fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.867004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972a954-ecb7-4670-8767-bf5f28641fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.889060 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35c87d-8ef5-4abc-85fd-c707918f677b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:44.889912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6972a954-ecb7-4670-8767-bf5f28641fa2 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:44.891120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6972a954-ecb7-4670-8767-bf5f28641fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:44.930722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7810-368b-4294-913a-a0caccf0f474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:44.931713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7810-368b-4294-913a-a0caccf0f474 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:44.932647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7810-368b-4294-913a-a0caccf0f474 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:44.932647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7810-368b-4294-913a-a0caccf0f474 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:44.933180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7810-368b-4294-913a-a0caccf0f474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:44.933514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7810-368b-4294-913a-a0caccf0f474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:44.934974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7810-368b-4294-913a-a0caccf0f474 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:44.952917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 363/749] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:44.952917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7810-368b-4294-913a-a0caccf0f474 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:44.965305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 387/750] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:44 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:16:44.966235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972a954-ecb7-4670-8767-bf5f28641fa2 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:45.022596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:45.074097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7810-368b-4294-913a-a0caccf0f474 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:45.074620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972a954-ecb7-4670-8767-bf5f28641fa2 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:45.109346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c2a3f-36d2-47f4-bf92-90de18238e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:45.109926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c2a3f-36d2-47f4-bf92-90de18238e07 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:45.110204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c2a3f-36d2-47f4-bf92-90de18238e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:45.110474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c2a3f-36d2-47f4-bf92-90de18238e07 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:45.111184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c2a3f-36d2-47f4-bf92-90de18238e07 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:45.111344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cea7be-0ba5-4b1c-a92c-a3077093b81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:45.111973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea7be-0ba5-4b1c-a92c-a3077093b81c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:45.112255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea7be-0ba5-4b1c-a92c-a3077093b81c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:45.112536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea7be-0ba5-4b1c-a92c-a3077093b81c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:45.112627 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c2a3f-36d2-47f4-bf92-90de18238e07 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:45.113407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cea7be-0ba5-4b1c-a92c-a3077093b81c None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=0bb6b4503a4e43f480af8430e7f4fc49) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:45.113724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cea7be-0ba5-4b1c-a92c-a3077093b81c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:45.115369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cea7be-0ba5-4b1c-a92c-a3077093b81c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:45.132906 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 364/751] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:44 2022] GET /identity/v3/role_assignments?user.id=0bb6b4503a4e43f480af8430e7f4fc49&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:45.132906 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cea7be-0ba5-4b1c-a92c-a3077093b81c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:45.136705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:45.138358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 388/752] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:16:44 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:16:45.139222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c2a3f-36d2-47f4-bf92-90de18238e07 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:45.352349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c2a3f-36d2-47f4-bf92-90de18238e07 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:45.358898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cea7be-0ba5-4b1c-a92c-a3077093b81c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:45.370843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:45.405571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcf114-3a50-41a6-a3b7-aa510732a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:45.406390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcf114-3a50-41a6-a3b7-aa510732a496 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:45.406678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcf114-3a50-41a6-a3b7-aa510732a496 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:45.406959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcf114-3a50-41a6-a3b7-aa510732a496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:45.411226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d056d32-dff1-41c5-b8a4-93059c36d483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:45.412026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d056d32-dff1-41c5-b8a4-93059c36d483 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:45.412307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d056d32-dff1-41c5-b8a4-93059c36d483 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:45.412722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d056d32-dff1-41c5-b8a4-93059c36d483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:45.419505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcf114-3a50-41a6-a3b7-aa510732a496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:45.421183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d056d32-dff1-41c5-b8a4-93059c36d483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:45.422655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d056d32-dff1-41c5-b8a4-93059c36d483 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-eedcf114-3a50-41a6-a3b7-aa510732a496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:45.427566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:45.428502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:45.429372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 389/753] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:45.437885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcf114-3a50-41a6-a3b7-aa510732a496 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-3d056d32-dff1-41c5-b8a4-93059c36d483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:45.441932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:45.443930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:45.446405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:45.446405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:45.446405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:45.446405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:45.446405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:45.446405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:45.446405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:45.446405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:45.446405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 365/754] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:45.450358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d056d32-dff1-41c5-b8a4-93059c36d483 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:45.522546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:45.564838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcf114-3a50-41a6-a3b7-aa510732a496 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:45.576542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d056d32-dff1-41c5-b8a4-93059c36d483 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:45.606401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdcfd0-a87d-4f68-bb26-8e97d87bdb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:45.608581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdcfd0-a87d-4f68-bb26-8e97d87bdb95 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:45.608790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdcfd0-a87d-4f68-bb26-8e97d87bdb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:45.609118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdcfd0-a87d-4f68-bb26-8e97d87bdb95 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:45.609890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdcfd0-a87d-4f68-bb26-8e97d87bdb95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:45.610207 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdcfd0-a87d-4f68-bb26-8e97d87bdb95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:45.611589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdcfd0-a87d-4f68-bb26-8e97d87bdb95 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:45.621807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 390/755] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:45.631697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32525126-a637-4927-954e-bc145cd3d0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:45.632419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32525126-a637-4927-954e-bc145cd3d0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:45.632702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32525126-a637-4927-954e-bc145cd3d0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:45.633002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32525126-a637-4927-954e-bc145cd3d0a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:45.633897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32525126-a637-4927-954e-bc145cd3d0a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:45.634359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32525126-a637-4927-954e-bc145cd3d0a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:45.635791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32525126-a637-4927-954e-bc145cd3d0a6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:45.639478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdcfd0-a87d-4f68-bb26-8e97d87bdb95 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:45.663893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 366/756] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:45.687052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32525126-a637-4927-954e-bc145cd3d0a6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:45.710411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:45.810455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdcfd0-a87d-4f68-bb26-8e97d87bdb95 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:45.853886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652a97f-c3d2-4188-a269-bc9e955e0b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:45.854695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652a97f-c3d2-4188-a269-bc9e955e0b79 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:45.855045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652a97f-c3d2-4188-a269-bc9e955e0b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:45.855323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652a97f-c3d2-4188-a269-bc9e955e0b79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:45.866856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652a97f-c3d2-4188-a269-bc9e955e0b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:45.869161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652a97f-c3d2-4188-a269-bc9e955e0b79 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-7652a97f-c3d2-4188-a269-bc9e955e0b79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:45.890501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:45.891546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:45.892433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:45.892433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:45.892433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:45.892433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:45.892433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 391/757] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:45.897408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652a97f-c3d2-4188-a269-bc9e955e0b79 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:45.900323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32525126-a637-4927-954e-bc145cd3d0a6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:45.951478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:45.974914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acb181-aa37-4a4c-be54-91d7beec5d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:45.975954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb181-aa37-4a4c-be54-91d7beec5d60 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:45.976436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb181-aa37-4a4c-be54-91d7beec5d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:45.977589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb181-aa37-4a4c-be54-91d7beec5d60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:45.978635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acb181-aa37-4a4c-be54-91d7beec5d60 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=ff6c489d950a4370bbf01866d2fc8fc3) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:45.979148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acb181-aa37-4a4c-be54-91d7beec5d60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:45.980789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acb181-aa37-4a4c-be54-91d7beec5d60 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:45.992278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 367/758] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:45 2022] GET /identity/v3/role_assignments?user.id=ff6c489d950a4370bbf01866d2fc8fc3&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:46.031836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:46.067584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652a97f-c3d2-4188-a269-bc9e955e0b79 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:46.124040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37f9b1-2ac6-41e0-9119-01847849fd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:46.124720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37f9b1-2ac6-41e0-9119-01847849fd38 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:46.125035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37f9b1-2ac6-41e0-9119-01847849fd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:46.125318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37f9b1-2ac6-41e0-9119-01847849fd38 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:46.126058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37f9b1-2ac6-41e0-9119-01847849fd38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:46.126368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37f9b1-2ac6-41e0-9119-01847849fd38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:46.127775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37f9b1-2ac6-41e0-9119-01847849fd38 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:46.148394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 392/759] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:46.154554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acb181-aa37-4a4c-be54-91d7beec5d60 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:46.328651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acb181-aa37-4a4c-be54-91d7beec5d60 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:46.373248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876bf45-f2a6-4600-bdc0-690dab200c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:46.374432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876bf45-f2a6-4600-bdc0-690dab200c9f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:46.374933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876bf45-f2a6-4600-bdc0-690dab200c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:46.375416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876bf45-f2a6-4600-bdc0-690dab200c9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:46.384555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876bf45-f2a6-4600-bdc0-690dab200c9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:46.386334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876bf45-f2a6-4600-bdc0-690dab200c9f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b876bf45-f2a6-4600-bdc0-690dab200c9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:46.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:46.409269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:46.410223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:46.412912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 368/760] 10.222.0.30 () {56 vars in 1093 bytes} [Mon May 23 15:16:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:46.418601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37f9b1-2ac6-41e0-9119-01847849fd38 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:46.569874 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37f9b1-2ac6-41e0-9119-01847849fd38 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:46.620759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0768b4-975f-44f2-a8f8-dbae2decdb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:46.621467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0768b4-975f-44f2-a8f8-dbae2decdb47 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:46.621749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0768b4-975f-44f2-a8f8-dbae2decdb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:46.622025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0768b4-975f-44f2-a8f8-dbae2decdb47 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:46.623134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0768b4-975f-44f2-a8f8-dbae2decdb47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:46.623440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0768b4-975f-44f2-a8f8-dbae2decdb47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:46.624854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0768b4-975f-44f2-a8f8-dbae2decdb47 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:46.649274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:46.653301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 393/761] 10.222.0.30 () {56 vars in 1133 bytes} [Mon May 23 15:16:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:46.658883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876bf45-f2a6-4600-bdc0-690dab200c9f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:46.724262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:46.799395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876bf45-f2a6-4600-bdc0-690dab200c9f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:46.853603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b74464-a2eb-41dc-a435-79dcd9553be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:46.854417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b74464-a2eb-41dc-a435-79dcd9553be5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:46.854702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b74464-a2eb-41dc-a435-79dcd9553be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:46.854981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b74464-a2eb-41dc-a435-79dcd9553be5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:46.873136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b74464-a2eb-41dc-a435-79dcd9553be5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:46.874609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b74464-a2eb-41dc-a435-79dcd9553be5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-16b74464-a2eb-41dc-a435-79dcd9553be5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:46.896259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:46.897246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:46.898169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:46.898169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:46.898169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:46.898169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:46.898169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 369/762] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:46.903888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0768b4-975f-44f2-a8f8-dbae2decdb47 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:47.050391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0768b4-975f-44f2-a8f8-dbae2decdb47 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:47.084486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2f6c8-986a-4382-91c1-ab31126ff2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:47.085179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2f6c8-986a-4382-91c1-ab31126ff2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:47.085456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2f6c8-986a-4382-91c1-ab31126ff2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:47.085727 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2f6c8-986a-4382-91c1-ab31126ff2b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:47.087111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2f6c8-986a-4382-91c1-ab31126ff2b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:47.087330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2f6c8-986a-4382-91c1-ab31126ff2b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:47.088788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2f6c8-986a-4382-91c1-ab31126ff2b2 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:47.100363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 394/763] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:47.105923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b74464-a2eb-41dc-a435-79dcd9553be5 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:47.267893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:47.272631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b74464-a2eb-41dc-a435-79dcd9553be5 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:47.324215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786d446-6084-48e4-9d87-fde3b6c2c5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:47.324979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786d446-6084-48e4-9d87-fde3b6c2c5df None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:47.325264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786d446-6084-48e4-9d87-fde3b6c2c5df None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:47.325538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786d446-6084-48e4-9d87-fde3b6c2c5df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:47.338549 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786d446-6084-48e4-9d87-fde3b6c2c5df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:47.340021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786d446-6084-48e4-9d87-fde3b6c2c5df None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b786d446-6084-48e4-9d87-fde3b6c2c5df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:47.360150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:47.361128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:47.361992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:47.361992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:47.361992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:47.361992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:47.361992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:47.361992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:47.361992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:47.361992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:47.361992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 370/764] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:47.367018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2f6c8-986a-4382-91c1-ab31126ff2b2 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:47.455702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cfc99-eb00-4a77-bf76-0e23c7411f73 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:47.456213 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cfc99-eb00-4a77-bf76-0e23c7411f73 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:47.456291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cfc99-eb00-4a77-bf76-0e23c7411f73 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:47.457641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 371/765] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:47.463689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:47.474560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad0804-37fe-4a9f-a101-06bf50fb18af None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:47.474897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad0804-37fe-4a9f-a101-06bf50fb18af None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:47.475107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad0804-37fe-4a9f-a101-06bf50fb18af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:47.502743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-02ad0804-37fe-4a9f-a101-06bf50fb18af None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:47.503966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-02ad0804-37fe-4a9f-a101-06bf50fb18af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:47.538871 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2f6c8-986a-4382-91c1-ab31126ff2b2 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:47.595311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b0235-cca5-43f1-8fb8-330ec070d83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:47.596763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b0235-cca5-43f1-8fb8-330ec070d83b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:47.597279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b0235-cca5-43f1-8fb8-330ec070d83b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:47.597758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b0235-cca5-43f1-8fb8-330ec070d83b None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:47.599069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b0235-cca5-43f1-8fb8-330ec070d83b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:47.600047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b0235-cca5-43f1-8fb8-330ec070d83b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:47.602878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b0235-cca5-43f1-8fb8-330ec070d83b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:47.626523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 395/766] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:47.634653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b0235-cca5-43f1-8fb8-330ec070d83b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:47.663920 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 372/767] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:47.689920 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad0804-37fe-4a9f-a101-06bf50fb18af None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:47.822810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:47.840440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad0804-37fe-4a9f-a101-06bf50fb18af None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:47.855631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b0235-cca5-43f1-8fb8-330ec070d83b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:47.890269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:47.901788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfd18d-3d10-4e38-b693-1e6e22137931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:47.903321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfd18d-3d10-4e38-b693-1e6e22137931 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:47.904003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfd18d-3d10-4e38-b693-1e6e22137931 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:47.906009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfd18d-3d10-4e38-b693-1e6e22137931 None admin] PATH_INFO: `/v3/services/network` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:47.907451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfd18d-3d10-4e38-b693-1e6e22137931 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:47.911121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfd18d-3d10-4e38-b693-1e6e22137931 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-15cfd18d-3d10-4e38-b693-1e6e22137931 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:47.928170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:47.929216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:16:47.930031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:47.935341 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41aab2-3ecb-4927-8005-c6a1615c44f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:47.936509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41aab2-3ecb-4927-8005-c6a1615c44f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:47.937023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41aab2-3ecb-4927-8005-c6a1615c44f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:47.937504 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41aab2-3ecb-4927-8005-c6a1615c44f9 None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/232cfaefaf584ac59720713e02cf2c2d/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:47.939708 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41aab2-3ecb-4927-8005-c6a1615c44f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=232cfaefaf584ac59720713e02cf2c2d, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:47.941415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41aab2-3ecb-4927-8005-c6a1615c44f9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:47.944960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 373/768] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:47 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:47.954051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfd18d-3d10-4e38-b693-1e6e22137931 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:47.966489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 396/769] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:16:47 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/232cfaefaf584ac59720713e02cf2c2d/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:47.977272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f56cf8-5dbe-4793-a38f-9d10da0821dc None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:47.977524 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f56cf8-5dbe-4793-a38f-9d10da0821dc None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:47.977766 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f56cf8-5dbe-4793-a38f-9d10da0821dc None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:47.983527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 397/770] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:16:47.985618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bfe81-50d4-435c-8fed-8515bd25e4b4 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:47.986083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bfe81-50d4-435c-8fed-8515bd25e4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:47.986426 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bfe81-50d4-435c-8fed-8515bd25e4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.013489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-dc6bfe81-50d4-435c-8fed-8515bd25e4b4 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:48.014682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-dc6bfe81-50d4-435c-8fed-8515bd25e4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:48.100565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfd18d-3d10-4e38-b693-1e6e22137931 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:48.111428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 398/771] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:16:48.113007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdedcf-5e0e-475a-937f-55c692f4a5aa None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.113301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdedcf-5e0e-475a-937f-55c692f4a5aa None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.113580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdedcf-5e0e-475a-937f-55c692f4a5aa None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.115438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 399/772] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:48.126453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38e78d-6b71-41d9-a84c-230e00cd0643 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.126854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38e78d-6b71-41d9-a84c-230e00cd0643 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.126985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38e78d-6b71-41d9-a84c-230e00cd0643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.144367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9cdd-c181-41c8-a90c-854712718d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:48.145097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9cdd-c181-41c8-a90c-854712718d1e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.145376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9cdd-c181-41c8-a90c-854712718d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.145648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9cdd-c181-41c8-a90c-854712718d1e None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.146373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9cdd-c181-41c8-a90c-854712718d1e None admin] RBAC: Adding query filter params (name=network) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:48.146676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9cdd-c181-41c8-a90c-854712718d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:48.148095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9cdd-c181-41c8-a90c-854712718d1e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:48.152483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4a38e78d-6b71-41d9-a84c-230e00cd0643 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:48.154087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4a38e78d-6b71-41d9-a84c-230e00cd0643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:48.154349 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 374/773] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:16:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:48.155303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9cdd-c181-41c8-a90c-854712718d1e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:48.241297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 400/774] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:48.242538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38e78d-6b71-41d9-a84c-230e00cd0643 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:48.296822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:48.300787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:48.361777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38e78d-6b71-41d9-a84c-230e00cd0643 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:48.390451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9cdd-c181-41c8-a90c-854712718d1e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:48.399304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc80dc0-9f2c-46f7-b9a4-1d91f91b02c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:48.400010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80dc0-9f2c-46f7-b9a4-1d91f91b02c7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.400299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80dc0-9f2c-46f7-b9a4-1d91f91b02c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.400581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80dc0-9f2c-46f7-b9a4-1d91f91b02c7 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.401362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80dc0-9f2c-46f7-b9a4-1d91f91b02c7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:48.401794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80dc0-9f2c-46f7-b9a4-1d91f91b02c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:48.403457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80dc0-9f2c-46f7-b9a4-1d91f91b02c7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:48.414617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 401/775] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:16:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:48.415572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc80dc0-9f2c-46f7-b9a4-1d91f91b02c7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:48.430761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852618f9-f003-4892-b1d0-01ae3830b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:48.431573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852618f9-f003-4892-b1d0-01ae3830b7cb None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.431849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852618f9-f003-4892-b1d0-01ae3830b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.432103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852618f9-f003-4892-b1d0-01ae3830b7cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.443324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852618f9-f003-4892-b1d0-01ae3830b7cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-852618f9-f003-4892-b1d0-01ae3830b7cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:48.451369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:48.452295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:48.453148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:48.457487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 375/776] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:48.458377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852618f9-f003-4892-b1d0-01ae3830b7cb None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:48.524672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc80dc0-9f2c-46f7-b9a4-1d91f91b02c7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:48.536704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:48.538123 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:48.564457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1691e-ae89-46ae-be67-b7fddfcbf597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:48.564673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852618f9-f003-4892-b1d0-01ae3830b7cb None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:48.565687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1691e-ae89-46ae-be67-b7fddfcbf597 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.566196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1691e-ae89-46ae-be67-b7fddfcbf597 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.566623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1691e-ae89-46ae-be67-b7fddfcbf597 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.567481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1691e-ae89-46ae-be67-b7fddfcbf597 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:48.569021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1691e-ae89-46ae-be67-b7fddfcbf597 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-15c1691e-ae89-46ae-be67-b7fddfcbf597 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:48.582622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:48.583638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:16:48.584453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:48.593360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 402/777] 10.222.0.30 () {56 vars in 1129 bytes} [Mon May 23 15:16:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:48.594582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1691e-ae89-46ae-be67-b7fddfcbf597 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:48.600112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c202cb6-d397-4b5f-9e69-3a53fb021d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:48.600774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c202cb6-d397-4b5f-9e69-3a53fb021d42 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.601080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c202cb6-d397-4b5f-9e69-3a53fb021d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.601352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c202cb6-d397-4b5f-9e69-3a53fb021d42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.602796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c202cb6-d397-4b5f-9e69-3a53fb021d42 None admin] RBAC: Adding query filter params (interface=public, service_id=4eaed6182c884247adf8e6afaccd8c56, region_id=RegionOne) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:48.603107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c202cb6-d397-4b5f-9e69-3a53fb021d42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:48.606746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c202cb6-d397-4b5f-9e69-3a53fb021d42 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:48.613357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 376/778] 10.222.0.30 () {56 vars in 1251 bytes} [Mon May 23 15:16:48 2022] GET /identity/v3/endpoints?service_id=4eaed6182c884247adf8e6afaccd8c56&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:48.614198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c202cb6-d397-4b5f-9e69-3a53fb021d42 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:48.714880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1691e-ae89-46ae-be67-b7fddfcbf597 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:48.716220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c202cb6-d397-4b5f-9e69-3a53fb021d42 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:48.748936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96d6bb-42f4-49ad-b1d4-0cf039a1bd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:48.749611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96d6bb-42f4-49ad-b1d4-0cf039a1bd86 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.749890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96d6bb-42f4-49ad-b1d4-0cf039a1bd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.750165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96d6bb-42f4-49ad-b1d4-0cf039a1bd86 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.750750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3446a-89ab-4bfa-8874-d5bb00955f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:48.750959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96d6bb-42f4-49ad-b1d4-0cf039a1bd86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:48.751215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96d6bb-42f4-49ad-b1d4-0cf039a1bd86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:48.751410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3446a-89ab-4bfa-8874-d5bb00955f3b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.751688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3446a-89ab-4bfa-8874-d5bb00955f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.751962 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3446a-89ab-4bfa-8874-d5bb00955f3b None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.752570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96d6bb-42f4-49ad-b1d4-0cf039a1bd86 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:48.752666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3446a-89ab-4bfa-8874-d5bb00955f3b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:48.752980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3446a-89ab-4bfa-8874-d5bb00955f3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:48.754532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3446a-89ab-4bfa-8874-d5bb00955f3b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:48.765989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 377/779] 10.222.0.30 () {56 vars in 1124 bytes} [Mon May 23 15:16:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:48.766152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 403/780] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:48.766152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96d6bb-42f4-49ad-b1d4-0cf039a1bd86 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:48.767036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9fdd3-3484-495a-b51b-2373dd4bd30c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.767319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9fdd3-3484-495a-b51b-2373dd4bd30c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.767593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9fdd3-3484-495a-b51b-2373dd4bd30c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.772902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 378/781] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:48.772902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9fdd3-3484-495a-b51b-2373dd4bd30c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:48.775969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:48.798543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:48.873676 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96d6bb-42f4-49ad-b1d4-0cf039a1bd86 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:48.889417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9fdd3-3484-495a-b51b-2373dd4bd30c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:48.909964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedde04-0624-46a6-add9-22b7bda14afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:48.910636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedde04-0624-46a6-add9-22b7bda14afb None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.910921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedde04-0624-46a6-add9-22b7bda14afb None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.911199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedde04-0624-46a6-add9-22b7bda14afb None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.911879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedde04-0624-46a6-add9-22b7bda14afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:48.913261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedde04-0624-46a6-add9-22b7bda14afb None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:48.924631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238035f-f280-4bbc-a467-e547f0c7b7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:48.925214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 404/782] 10.222.0.30 () {56 vars in 1087 bytes} [Mon May 23 15:16:48 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:48.925660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238035f-f280-4bbc-a467-e547f0c7b7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.925939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238035f-f280-4bbc-a467-e547f0c7b7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.926043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedde04-0624-46a6-add9-22b7bda14afb None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:48.926209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238035f-f280-4bbc-a467-e547f0c7b7e7 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.926995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238035f-f280-4bbc-a467-e547f0c7b7e7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:48.927394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238035f-f280-4bbc-a467-e547f0c7b7e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:48.928911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238035f-f280-4bbc-a467-e547f0c7b7e7 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:48.944413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 379/783] 10.222.0.30 () {56 vars in 1124 bytes} [Mon May 23 15:16:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:48.946837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44a495-ed36-4bd2-b53d-9b6859d78536 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:48.947582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44a495-ed36-4bd2-b53d-9b6859d78536 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:48.948154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44a495-ed36-4bd2-b53d-9b6859d78536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:48.968031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-6c44a495-ed36-4bd2-b53d-9b6859d78536 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:48.969767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6c44a495-ed36-4bd2-b53d-9b6859d78536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:49.041361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:49.047672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:49.050199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedde04-0624-46a6-add9-22b7bda14afb None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:49.065306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 380/784] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:49.066726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44a495-ed36-4bd2-b53d-9b6859d78536 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:49.088826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b8c42-ee13-4951-92b7-adcf13a99382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:49.089669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b8c42-ee13-4951-92b7-adcf13a99382 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:49.089947 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b8c42-ee13-4951-92b7-adcf13a99382 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:49.090224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b8c42-ee13-4951-92b7-adcf13a99382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:49.102839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b8c42-ee13-4951-92b7-adcf13a99382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:49.103963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b8c42-ee13-4951-92b7-adcf13a99382 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-df5b8c42-ee13-4951-92b7-adcf13a99382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:49.127052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:49.127968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:49.128861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:49.128861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:49.128861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:49.128861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:49.128861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 405/785] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:49.129718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b8c42-ee13-4951-92b7-adcf13a99382 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:49.213962 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44a495-ed36-4bd2-b53d-9b6859d78536 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:49.249921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b8c42-ee13-4951-92b7-adcf13a99382 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:49.259406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b40f1-f895-43dd-8b4a-2c38372cd2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:49.260196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b40f1-f895-43dd-8b4a-2c38372cd2cc None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:49.260511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b40f1-f895-43dd-8b4a-2c38372cd2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:49.261217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b40f1-f895-43dd-8b4a-2c38372cd2cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:49.262035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b40f1-f895-43dd-8b4a-2c38372cd2cc None admin] RBAC: Adding query filter params (interface=public, service_id=500d77b1a2f846c4ba207eefdaac6e18, region_id=RegionOne) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:49.262369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b40f1-f895-43dd-8b4a-2c38372cd2cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:49.264625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b40f1-f895-43dd-8b4a-2c38372cd2cc None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:49.272435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 381/786] 10.222.0.30 () {56 vars in 1251 bytes} [Mon May 23 15:16:49 2022] GET /identity/v3/endpoints?service_id=500d77b1a2f846c4ba207eefdaac6e18&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:49.277441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b40f1-f895-43dd-8b4a-2c38372cd2cc None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:49.293511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3900da9-71c6-4424-9123-48be71c87f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:49.294673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3900da9-71c6-4424-9123-48be71c87f17 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:49.295185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3900da9-71c6-4424-9123-48be71c87f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:49.295660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3900da9-71c6-4424-9123-48be71c87f17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:49.307696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3900da9-71c6-4424-9123-48be71c87f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-d3900da9-71c6-4424-9123-48be71c87f17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:49.315800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:49.316751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:49.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:49.324971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 406/787] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:49.326127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3900da9-71c6-4424-9123-48be71c87f17 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:49.336267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:49.359100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:49.403173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b40f1-f895-43dd-8b4a-2c38372cd2cc None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:49.438618 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993f064-77df-4eb1-8a7c-8175e1603c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:49.439286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993f064-77df-4eb1-8a7c-8175e1603c83 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:49.439566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993f064-77df-4eb1-8a7c-8175e1603c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:49.439839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993f064-77df-4eb1-8a7c-8175e1603c83 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:49.440571 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993f064-77df-4eb1-8a7c-8175e1603c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:49.441107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993f064-77df-4eb1-8a7c-8175e1603c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:49.442272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993f064-77df-4eb1-8a7c-8175e1603c83 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:49.459633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 382/788] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:49.461649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993f064-77df-4eb1-8a7c-8175e1603c83 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:49.469321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3900da9-71c6-4424-9123-48be71c87f17 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:49.505517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105494fd-215e-4396-b135-321b5eff316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:49.506217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105494fd-215e-4396-b135-321b5eff316f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:49.506503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105494fd-215e-4396-b135-321b5eff316f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:49.506779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105494fd-215e-4396-b135-321b5eff316f None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:49.507468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105494fd-215e-4396-b135-321b5eff316f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:49.509708 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105494fd-215e-4396-b135-321b5eff316f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:49.515312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 407/789] 10.222.0.30 () {56 vars in 1087 bytes} [Mon May 23 15:16:49 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:49.516316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105494fd-215e-4396-b135-321b5eff316f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:49.656975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993f064-77df-4eb1-8a7c-8175e1603c83 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:49.669212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:49.706148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:49.723478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105494fd-215e-4396-b135-321b5eff316f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:49.729968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3621b5e-5430-42ca-897e-50b283b0299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:49.730875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621b5e-5430-42ca-897e-50b283b0299c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:49.735892 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621b5e-5430-42ca-897e-50b283b0299c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:49.736326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621b5e-5430-42ca-897e-50b283b0299c None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:49.737219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3621b5e-5430-42ca-897e-50b283b0299c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:49.737530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3621b5e-5430-42ca-897e-50b283b0299c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:49.738945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3621b5e-5430-42ca-897e-50b283b0299c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:49.760082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 383/790] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:49.761134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3621b5e-5430-42ca-897e-50b283b0299c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:49.804058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df54050-03ec-48d8-836e-fe9697ab340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:49.805642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df54050-03ec-48d8-836e-fe9697ab340b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:49.805642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df54050-03ec-48d8-836e-fe9697ab340b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:49.805642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df54050-03ec-48d8-836e-fe9697ab340b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:49.815151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df54050-03ec-48d8-836e-fe9697ab340b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:49.817109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df54050-03ec-48d8-836e-fe9697ab340b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-7df54050-03ec-48d8-836e-fe9697ab340b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:49.838913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:49.839922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:49.840852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 408/791] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:16:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:49.849925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df54050-03ec-48d8-836e-fe9697ab340b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:49.908761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3621b5e-5430-42ca-897e-50b283b0299c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:49.950675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6879f7-e8d2-46e7-a53d-01b28880e8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:49.951886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6879f7-e8d2-46e7-a53d-01b28880e8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:49.952405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6879f7-e8d2-46e7-a53d-01b28880e8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:49.952948 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6879f7-e8d2-46e7-a53d-01b28880e8e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:49.975879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6879f7-e8d2-46e7-a53d-01b28880e8e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:49.979714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6879f7-e8d2-46e7-a53d-01b28880e8e0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-5e6879f7-e8d2-46e7-a53d-01b28880e8e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:50.003504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:50.004509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:50.005485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:50.005485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:50.005485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:50.005485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:50.007910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 384/792] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:50.013225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6879f7-e8d2-46e7-a53d-01b28880e8e0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:50.060758 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:50.131602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6879f7-e8d2-46e7-a53d-01b28880e8e0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:50.184464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df54050-03ec-48d8-836e-fe9697ab340b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:50.186724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e28ca2-97ab-466e-a351-35c0ca53c605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:50.187620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e28ca2-97ab-466e-a351-35c0ca53c605 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:50.187903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e28ca2-97ab-466e-a351-35c0ca53c605 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:50.188198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e28ca2-97ab-466e-a351-35c0ca53c605 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:50.188981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e28ca2-97ab-466e-a351-35c0ca53c605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:50.189346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e28ca2-97ab-466e-a351-35c0ca53c605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:50.190877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e28ca2-97ab-466e-a351-35c0ca53c605 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:50.224385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 385/793] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:50.231308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e28ca2-97ab-466e-a351-35c0ca53c605 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:50.252277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:50.271503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd59d79-afc9-4f6d-9b45-207fc635eb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:50.272263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59d79-afc9-4f6d-9b45-207fc635eb6e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:50.273908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59d79-afc9-4f6d-9b45-207fc635eb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:50.274309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59d79-afc9-4f6d-9b45-207fc635eb6e None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:50.275178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd59d79-afc9-4f6d-9b45-207fc635eb6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:50.275492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd59d79-afc9-4f6d-9b45-207fc635eb6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:50.277035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd59d79-afc9-4f6d-9b45-207fc635eb6e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:50.320796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 409/794] 10.222.0.30 () {56 vars in 1149 bytes} [Mon May 23 15:16:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:50.325956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd59d79-afc9-4f6d-9b45-207fc635eb6e None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:50.430439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:50.439507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e28ca2-97ab-466e-a351-35c0ca53c605 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:50.481330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd59d79-afc9-4f6d-9b45-207fc635eb6e None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:50.481526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf0dd8-e987-43fd-81b6-9134a971f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:50.482185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf0dd8-e987-43fd-81b6-9134a971f5aa None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:50.482491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf0dd8-e987-43fd-81b6-9134a971f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:50.482877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf0dd8-e987-43fd-81b6-9134a971f5aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:50.492178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf0dd8-e987-43fd-81b6-9134a971f5aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:50.494017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf0dd8-e987-43fd-81b6-9134a971f5aa None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-becf0dd8-e987-43fd-81b6-9134a971f5aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:50.517377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:50.518343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:50.520011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 386/795] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:50.520582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce91b3-e591-4ffe-96e8-687d042d7bb7 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:50.520824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce91b3-e591-4ffe-96e8-687d042d7bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:50.522571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15b5a9-e2d0-43fa-9d6d-d223e950a88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:50.522819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce91b3-e591-4ffe-96e8-687d042d7bb7 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:50.522991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b5a9-e2d0-43fa-9d6d-d223e950a88b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:50.522991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b5a9-e2d0-43fa-9d6d-d223e950a88b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:50.522991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b5a9-e2d0-43fa-9d6d-d223e950a88b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:50.523383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 387/796] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:50.525766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce91b3-e591-4ffe-96e8-687d042d7bb7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:50.539551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15b5a9-e2d0-43fa-9d6d-d223e950a88b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:50.542267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15b5a9-e2d0-43fa-9d6d-d223e950a88b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-ff15b5a9-e2d0-43fa-9d6d-d223e950a88b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:50.563184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:50.565322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:50.567343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:50.567343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:50.567343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:50.567343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:50.567343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 410/797] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:50.567343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9681d-679c-4c4b-b475-17478bc22650 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:50.567343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9681d-679c-4c4b-b475-17478bc22650 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:50.567343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9681d-679c-4c4b-b475-17478bc22650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:50.585162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-87c9681d-679c-4c4b-b475-17478bc22650 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:50.587810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-87c9681d-679c-4c4b-b475-17478bc22650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:50.639619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce91b3-e591-4ffe-96e8-687d042d7bb7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:50.645470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:50.676685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e262ee-0e35-4602-b279-c93e8da8adb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:50.677074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e262ee-0e35-4602-b279-c93e8da8adb2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:50.677322 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e262ee-0e35-4602-b279-c93e8da8adb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:50.677599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e262ee-0e35-4602-b279-c93e8da8adb2 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:50.678577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e262ee-0e35-4602-b279-c93e8da8adb2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:50.678915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e262ee-0e35-4602-b279-c93e8da8adb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:50.681016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e262ee-0e35-4602-b279-c93e8da8adb2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:50.708989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 388/799] 10.222.0.30 () {56 vars in 1135 bytes} [Mon May 23 15:16:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:50.708989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e262ee-0e35-4602-b279-c93e8da8adb2 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:50.709256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 411/799] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:50.712282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9681d-679c-4c4b-b475-17478bc22650 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:50.729359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:50.826864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e262ee-0e35-4602-b279-c93e8da8adb2 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:50.863462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9681d-679c-4c4b-b475-17478bc22650 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:50.865481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc00bd2-2705-49b9-83a2-8940c6176244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:50.866163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc00bd2-2705-49b9-83a2-8940c6176244 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:50.866468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc00bd2-2705-49b9-83a2-8940c6176244 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:50.866720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc00bd2-2705-49b9-83a2-8940c6176244 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:50.867636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc00bd2-2705-49b9-83a2-8940c6176244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:50.868560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc00bd2-2705-49b9-83a2-8940c6176244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:50.870109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc00bd2-2705-49b9-83a2-8940c6176244 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:50.885757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 389/800] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:50.886935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc00bd2-2705-49b9-83a2-8940c6176244 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:50.916161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b3730-78a6-425e-8e67-fa64c7972cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:50.917937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b3730-78a6-425e-8e67-fa64c7972cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:50.918424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b3730-78a6-425e-8e67-fa64c7972cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:50.918914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b3730-78a6-425e-8e67-fa64c7972cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:50.933127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b3730-78a6-425e-8e67-fa64c7972cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:50.934976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b3730-78a6-425e-8e67-fa64c7972cb1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-df1b3730-78a6-425e-8e67-fa64c7972cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:50.958134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:50.959395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:50.960350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:50.960350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:50.960350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:50.960350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:50.962952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 412/801] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:50.964130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b3730-78a6-425e-8e67-fa64c7972cb1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:50.968432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:50.998308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc00bd2-2705-49b9-83a2-8940c6176244 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:50.998883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:51.038750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13c7a9-fd57-45ed-a6db-770c70bee217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:51.039591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13c7a9-fd57-45ed-a6db-770c70bee217 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.040408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13c7a9-fd57-45ed-a6db-770c70bee217 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.040694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13c7a9-fd57-45ed-a6db-770c70bee217 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.052003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13c7a9-fd57-45ed-a6db-770c70bee217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-2b13c7a9-fd57-45ed-a6db-770c70bee217 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:51.060543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:51.061575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:51.062582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:51.066688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 390/802] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:51.067532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13c7a9-fd57-45ed-a6db-770c70bee217 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:51.108172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b3730-78a6-425e-8e67-fa64c7972cb1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:51.154566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc2541-8c31-4fe7-80be-807a86a57b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:51.155898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc2541-8c31-4fe7-80be-807a86a57b6e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.156423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc2541-8c31-4fe7-80be-807a86a57b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.157005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc2541-8c31-4fe7-80be-807a86a57b6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.165839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc2541-8c31-4fe7-80be-807a86a57b6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:51.167682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc2541-8c31-4fe7-80be-807a86a57b6e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:51.177759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13c7a9-fd57-45ed-a6db-770c70bee217 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-ebdc2541-8c31-4fe7-80be-807a86a57b6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:51.185133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:51.186112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:51.187145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:51.187145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:51.187145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:51.187145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:51.187145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:51.187145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:51.187145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:51.187145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:51.189981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 413/803] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:51.191160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc2541-8c31-4fe7-80be-807a86a57b6e None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:51.215778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b096f2c-13e4-418a-9d33-c4585718563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:51.216382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b096f2c-13e4-418a-9d33-c4585718563c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.216659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b096f2c-13e4-418a-9d33-c4585718563c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.217004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b096f2c-13e4-418a-9d33-c4585718563c None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.217735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b096f2c-13e4-418a-9d33-c4585718563c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:51.218087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b096f2c-13e4-418a-9d33-c4585718563c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:51.219466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b096f2c-13e4-418a-9d33-c4585718563c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:51.221154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:51.238010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 391/804] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:51.238767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b096f2c-13e4-418a-9d33-c4585718563c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:51.294711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:51.318142 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc2541-8c31-4fe7-80be-807a86a57b6e None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:51.356716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b096f2c-13e4-418a-9d33-c4585718563c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:51.369070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7012a-339e-4172-95a1-70186126f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:51.369975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7012a-339e-4172-95a1-70186126f113 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.370388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7012a-339e-4172-95a1-70186126f113 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.370706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7012a-339e-4172-95a1-70186126f113 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.371693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7012a-339e-4172-95a1-70186126f113 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:51.372026 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7012a-339e-4172-95a1-70186126f113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:51.373995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7012a-339e-4172-95a1-70186126f113 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:51.388733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 414/805] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:51.388733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7012a-339e-4172-95a1-70186126f113 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:51.400746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82653fe4-124f-45eb-8137-713822533a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:51.401485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82653fe4-124f-45eb-8137-713822533a4b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.401769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82653fe4-124f-45eb-8137-713822533a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.402063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82653fe4-124f-45eb-8137-713822533a4b None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.402883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82653fe4-124f-45eb-8137-713822533a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:51.403217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82653fe4-124f-45eb-8137-713822533a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:51.404730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82653fe4-124f-45eb-8137-713822533a4b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:51.418683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 392/806] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:51.420726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cdcfd-0011-40c0-8394-8e496b43d75f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.421071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cdcfd-0011-40c0-8394-8e496b43d75f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.421373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cdcfd-0011-40c0-8394-8e496b43d75f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.423793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 393/807] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:51.424476 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cdcfd-0011-40c0-8394-8e496b43d75f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:51.489182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:51.498231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7012a-339e-4172-95a1-70186126f113 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:51.537108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee826e-a97c-491b-81b8-5aa5be214133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:51.537369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cdcfd-0011-40c0-8394-8e496b43d75f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:51.537910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee826e-a97c-491b-81b8-5aa5be214133 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.538192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee826e-a97c-491b-81b8-5aa5be214133 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.538465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee826e-a97c-491b-81b8-5aa5be214133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.546162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee826e-a97c-491b-81b8-5aa5be214133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:51.547964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee826e-a97c-491b-81b8-5aa5be214133 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-fbee826e-a97c-491b-81b8-5aa5be214133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:51.563643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:51.564683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 415/808] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:51.566019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee826e-a97c-491b-81b8-5aa5be214133 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:51.571197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:51.578988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe68a6-32a6-4912-85b0-a547cae744db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:51.579740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe68a6-32a6-4912-85b0-a547cae744db None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.580092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe68a6-32a6-4912-85b0-a547cae744db None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.580294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe68a6-32a6-4912-85b0-a547cae744db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.592960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe68a6-32a6-4912-85b0-a547cae744db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:51.594431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe68a6-32a6-4912-85b0-a547cae744db None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-4bbe68a6-32a6-4912-85b0-a547cae744db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:51.614147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:51.615034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:51.615921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:51.615921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:51.615921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:51.615921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:51.620606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 394/809] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:51.621820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61462bff-7666-4daa-94c9-913a188f357f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.622110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61462bff-7666-4daa-94c9-913a188f357f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.622390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61462bff-7666-4daa-94c9-913a188f357f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.640549 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-61462bff-7666-4daa-94c9-913a188f357f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:51.642010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-61462bff-7666-4daa-94c9-913a188f357f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:51.703020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee826e-a97c-491b-81b8-5aa5be214133 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:51.726091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 395/810] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:51.727897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61462bff-7666-4daa-94c9-913a188f357f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:51.755036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89383b0-b2d7-48e4-aece-7ada88247de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:51.756442 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89383b0-b2d7-48e4-aece-7ada88247de1 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.756731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89383b0-b2d7-48e4-aece-7ada88247de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.757040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89383b0-b2d7-48e4-aece-7ada88247de1 None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/0bb6b4503a4e43f480af8430e7f4fc49/roles/2c4cf4c4707947848a30121234a5157c` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.766104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:51.775362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89383b0-b2d7-48e4-aece-7ada88247de1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=0bb6b4503a4e43f480af8430e7f4fc49, role_id=2c4cf4c4707947848a30121234a5157c)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:51.777444 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89383b0-b2d7-48e4-aece-7ada88247de1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:51.804905 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 416/811] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:16:51 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/0bb6b4503a4e43f480af8430e7f4fc49/roles/2c4cf4c4707947848a30121234a5157c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:51.804905 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89383b0-b2d7-48e4-aece-7ada88247de1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:51.850183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61462bff-7666-4daa-94c9-913a188f357f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:51.878079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:51.897527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b5b2c-aa4b-4b77-b602-0c9b9bb5ea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:51.898818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b5b2c-aa4b-4b77-b602-0c9b9bb5ea44 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:51.899346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b5b2c-aa4b-4b77-b602-0c9b9bb5ea44 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:51.900424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b5b2c-aa4b-4b77-b602-0c9b9bb5ea44 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:51.901760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b5b2c-aa4b-4b77-b602-0c9b9bb5ea44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:51.902338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b5b2c-aa4b-4b77-b602-0c9b9bb5ea44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:51.905367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b5b2c-aa4b-4b77-b602-0c9b9bb5ea44 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:51.923348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 396/812] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:51.924796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b5b2c-aa4b-4b77-b602-0c9b9bb5ea44 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:51.948120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89383b0-b2d7-48e4-aece-7ada88247de1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.003335 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fa118-ef68-4826-acb4-8ef0cef2006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.004012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fa118-ef68-4826-acb4-8ef0cef2006f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.004842 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fa118-ef68-4826-acb4-8ef0cef2006f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.004842 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fa118-ef68-4826-acb4-8ef0cef2006f None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.005362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fa118-ef68-4826-acb4-8ef0cef2006f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:52.005619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fa118-ef68-4826-acb4-8ef0cef2006f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.007166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fa118-ef68-4826-acb4-8ef0cef2006f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:52.023713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 417/813] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:52.023713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fa118-ef68-4826-acb4-8ef0cef2006f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:52.053956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:52.061977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b5b2c-aa4b-4b77-b602-0c9b9bb5ea44 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.099139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531a5da-e061-4ec6-9692-9334b1d72a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.099972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531a5da-e061-4ec6-9692-9334b1d72a8e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.100253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531a5da-e061-4ec6-9692-9334b1d72a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.100530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531a5da-e061-4ec6-9692-9334b1d72a8e None admin] PATH_INFO: `/v3/services/network` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.101225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531a5da-e061-4ec6-9692-9334b1d72a8e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.103331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531a5da-e061-4ec6-9692-9334b1d72a8e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-4531a5da-e061-4ec6-9692-9334b1d72a8e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:52.115509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:52.116675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:52.118191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 397/814] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:51 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:52.119057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d5709-24a9-42c8-b421-5dd6c04d918c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.119057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d5709-24a9-42c8-b421-5dd6c04d918c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.119057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d5709-24a9-42c8-b421-5dd6c04d918c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.124642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 398/815] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:52.124642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d5709-24a9-42c8-b421-5dd6c04d918c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:52.140677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fa118-ef68-4826-acb4-8ef0cef2006f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.155505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:52.177462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0666da-8bd8-481b-b5b1-1ac73ee91c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.178384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0666da-8bd8-481b-b5b1-1ac73ee91c85 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.178663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0666da-8bd8-481b-b5b1-1ac73ee91c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.178932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0666da-8bd8-481b-b5b1-1ac73ee91c85 None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/ff6c489d950a4370bbf01866d2fc8fc3/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.181735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0666da-8bd8-481b-b5b1-1ac73ee91c85 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=ff6c489d950a4370bbf01866d2fc8fc3, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.183186 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0666da-8bd8-481b-b5b1-1ac73ee91c85 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:52.209407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 418/816] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:16:52 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/ff6c489d950a4370bbf01866d2fc8fc3/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:16:52.210529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0666da-8bd8-481b-b5b1-1ac73ee91c85 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:52.242547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d5709-24a9-42c8-b421-5dd6c04d918c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.289391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be16054-23af-408c-a234-5f498923f843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.290767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be16054-23af-408c-a234-5f498923f843 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.291089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be16054-23af-408c-a234-5f498923f843 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.291365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be16054-23af-408c-a234-5f498923f843 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.303648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be16054-23af-408c-a234-5f498923f843 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.306056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be16054-23af-408c-a234-5f498923f843 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-2be16054-23af-408c-a234-5f498923f843 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:52.327033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:52.328073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:16:52.329046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:52.333330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 399/817] 10.222.0.30 () {56 vars in 1093 bytes} [Mon May 23 15:16:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:52.335935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee214-84f9-429c-a8a7-48a42f5cbe14 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.336272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee214-84f9-429c-a8a7-48a42f5cbe14 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.337080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee214-84f9-429c-a8a7-48a42f5cbe14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.355132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0666da-8bd8-481b-b5b1-1ac73ee91c85 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.359535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-2feee214-84f9-429c-a8a7-48a42f5cbe14 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:52.360762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-2feee214-84f9-429c-a8a7-48a42f5cbe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:52.382953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:52.409805 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cb55e-dd2c-45e5-b294-108d026da91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.410491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cb55e-dd2c-45e5-b294-108d026da91f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.410775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cb55e-dd2c-45e5-b294-108d026da91f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.411050 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cb55e-dd2c-45e5-b294-108d026da91f None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.411770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439cb55e-dd2c-45e5-b294-108d026da91f None admin] RBAC: Adding query filter params (name=network) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:52.412076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439cb55e-dd2c-45e5-b294-108d026da91f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.413557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439cb55e-dd2c-45e5-b294-108d026da91f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:52.420406 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 419/818] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:16:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:52.425703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cb55e-dd2c-45e5-b294-108d026da91f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:52.462072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 400/819] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:52.463469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feee214-84f9-429c-a8a7-48a42f5cbe14 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:52.469264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:52.550021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cb55e-dd2c-45e5-b294-108d026da91f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.596051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2991a1f-c85d-4325-9829-69da4378fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.596767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2991a1f-c85d-4325-9829-69da4378fa94 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.597090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2991a1f-c85d-4325-9829-69da4378fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.597474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2991a1f-c85d-4325-9829-69da4378fa94 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.598533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2991a1f-c85d-4325-9829-69da4378fa94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:52.598847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2991a1f-c85d-4325-9829-69da4378fa94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.600266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2991a1f-c85d-4325-9829-69da4378fa94 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:52.601205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feee214-84f9-429c-a8a7-48a42f5cbe14 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.625272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 420/820] 10.222.0.30 () {56 vars in 1133 bytes} [Mon May 23 15:16:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:52.626294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2991a1f-c85d-4325-9829-69da4378fa94 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:52.644613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950b50e-3f2e-4f5e-86cb-39be51336499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.645623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950b50e-3f2e-4f5e-86cb-39be51336499 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.646058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950b50e-3f2e-4f5e-86cb-39be51336499 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.646513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950b50e-3f2e-4f5e-86cb-39be51336499 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.647369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950b50e-3f2e-4f5e-86cb-39be51336499 None admin] RBAC: Adding query filter params (type=network) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:52.647890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950b50e-3f2e-4f5e-86cb-39be51336499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.650326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950b50e-3f2e-4f5e-86cb-39be51336499 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:52.656325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 401/821] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:16:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:52.661925 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950b50e-3f2e-4f5e-86cb-39be51336499 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:52.678648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:52.735801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:52.743878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2991a1f-c85d-4325-9829-69da4378fa94 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.779892 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950b50e-3f2e-4f5e-86cb-39be51336499 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.783125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cded0f-c8ae-426d-96b0-7e12e524dda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.784094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cded0f-c8ae-426d-96b0-7e12e524dda1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.784521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cded0f-c8ae-426d-96b0-7e12e524dda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.784989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cded0f-c8ae-426d-96b0-7e12e524dda1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.785786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cded0f-c8ae-426d-96b0-7e12e524dda1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.787768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cded0f-c8ae-426d-96b0-7e12e524dda1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-14cded0f-c8ae-426d-96b0-7e12e524dda1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:52.801988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:52.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:16:52.804070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:52.812108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 421/822] 10.222.0.30 () {56 vars in 1129 bytes} [Mon May 23 15:16:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:52.813359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cded0f-c8ae-426d-96b0-7e12e524dda1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:52.815423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660884f-49ac-4047-98df-49337d6c78c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.816068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660884f-49ac-4047-98df-49337d6c78c7 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.816347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660884f-49ac-4047-98df-49337d6c78c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.816619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660884f-49ac-4047-98df-49337d6c78c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.886834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660884f-49ac-4047-98df-49337d6c78c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.888313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660884f-49ac-4047-98df-49337d6c78c7 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:52.904171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:52.929185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cded0f-c8ae-426d-96b0-7e12e524dda1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:52.968120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58067353-9057-4a2b-b16d-e5d472e16c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:52.969056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58067353-9057-4a2b-b16d-e5d472e16c4e None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:52.969577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58067353-9057-4a2b-b16d-e5d472e16c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:52.970021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58067353-9057-4a2b-b16d-e5d472e16c4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:52.971002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58067353-9057-4a2b-b16d-e5d472e16c4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:52.972512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58067353-9057-4a2b-b16d-e5d472e16c4e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:53.002996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 402/823] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:16:53.002996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660884f-49ac-4047-98df-49337d6c78c7 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:53.005046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 422/824] 10.222.0.30 () {60 vars in 1144 bytes} [Mon May 23 15:16:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:16:53.006598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58067353-9057-4a2b-b16d-e5d472e16c4e None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:53.029931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:53.157276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660884f-49ac-4047-98df-49337d6c78c7 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:53.161505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58067353-9057-4a2b-b16d-e5d472e16c4e None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:53.172916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:53.200320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb44c00-49f9-4246-b7a4-24460d0e1477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:53.200320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb44c00-49f9-4246-b7a4-24460d0e1477 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:53.200320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb44c00-49f9-4246-b7a4-24460d0e1477 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:53.200320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb44c00-49f9-4246-b7a4-24460d0e1477 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:53.200742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb44c00-49f9-4246-b7a4-24460d0e1477 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:53.200742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb44c00-49f9-4246-b7a4-24460d0e1477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:53.200742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb44c00-49f9-4246-b7a4-24460d0e1477 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:53.204781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 403/825] 10.222.0.30 () {56 vars in 1124 bytes} [Mon May 23 15:16:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:53.214840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb44c00-49f9-4246-b7a4-24460d0e1477 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:53.238436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cac9d9-0e64-40bd-8b30-f9162491595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:53.239538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cac9d9-0e64-40bd-8b30-f9162491595c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:53.240064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cac9d9-0e64-40bd-8b30-f9162491595c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:53.240643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cac9d9-0e64-40bd-8b30-f9162491595c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:53.253483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cac9d9-0e64-40bd-8b30-f9162491595c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:53.255180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cac9d9-0e64-40bd-8b30-f9162491595c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-49cac9d9-0e64-40bd-8b30-f9162491595c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:53.275282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:53.276408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:53.277503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:53.277503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:53.277503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:53.277503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:53.283895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 423/826] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:53.285467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cac9d9-0e64-40bd-8b30-f9162491595c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:53.392192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:53.409227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb44c00-49f9-4246-b7a4-24460d0e1477 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:53.444476 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cac9d9-0e64-40bd-8b30-f9162491595c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:53.462607 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb67d1-f2db-4230-83a4-15305785ef41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:53.464954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb67d1-f2db-4230-83a4-15305785ef41 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:53.465414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb67d1-f2db-4230-83a4-15305785ef41 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:53.465832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb67d1-f2db-4230-83a4-15305785ef41 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:53.466805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb67d1-f2db-4230-83a4-15305785ef41 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:53.467260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb67d1-f2db-4230-83a4-15305785ef41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:53.468816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb67d1-f2db-4230-83a4-15305785ef41 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:53.477036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 404/827] 10.222.0.30 () {56 vars in 1124 bytes} [Mon May 23 15:16:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:53.482207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb67d1-f2db-4230-83a4-15305785ef41 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:53.498673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a4134-8165-4064-9439-ceae4028879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:53.499439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a4134-8165-4064-9439-ceae4028879b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:53.499791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a4134-8165-4064-9439-ceae4028879b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:53.500132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a4134-8165-4064-9439-ceae4028879b None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:53.500946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a4134-8165-4064-9439-ceae4028879b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:53.501321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a4134-8165-4064-9439-ceae4028879b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:53.502765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a4134-8165-4064-9439-ceae4028879b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:53.521298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 424/828] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:53.529974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a4134-8165-4064-9439-ceae4028879b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:53.551223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:53.627768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb67d1-f2db-4230-83a4-15305785ef41 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:53.658509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a4134-8165-4064-9439-ceae4028879b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:53.665271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc184103-5a49-4cda-8312-1b380e0379d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:53.666258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc184103-5a49-4cda-8312-1b380e0379d7 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:53.666737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc184103-5a49-4cda-8312-1b380e0379d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:53.667208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc184103-5a49-4cda-8312-1b380e0379d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:53.668149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc184103-5a49-4cda-8312-1b380e0379d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:53.669747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc184103-5a49-4cda-8312-1b380e0379d7 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:53.687334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:53.695266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 405/829] 10.222.0.30 () {60 vars in 1144 bytes} [Mon May 23 15:16:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:16:53.705894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca58e0-a335-4e68-b30a-51e33ddfbee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:53.706706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca58e0-a335-4e68-b30a-51e33ddfbee8 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:53.706984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca58e0-a335-4e68-b30a-51e33ddfbee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:53.707256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca58e0-a335-4e68-b30a-51e33ddfbee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:53.715835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca58e0-a335-4e68-b30a-51e33ddfbee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:53.717392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca58e0-a335-4e68-b30a-51e33ddfbee8 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-42ca58e0-a335-4e68-b30a-51e33ddfbee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:53.733239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:53.734216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:53.735225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:53.735225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:53.735225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:53.735225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:53.735225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:53.735225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:53.735225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:53.735225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:53.738919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 425/830] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:53.740773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc184103-5a49-4cda-8312-1b380e0379d7 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:53.858832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:53.863294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc184103-5a49-4cda-8312-1b380e0379d7 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:53.904495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6e5d2-7899-4400-b4df-60c38540d0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:53.905209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6e5d2-7899-4400-b4df-60c38540d0ea None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:53.905498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6e5d2-7899-4400-b4df-60c38540d0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:53.905773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6e5d2-7899-4400-b4df-60c38540d0ea None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:53.906574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6e5d2-7899-4400-b4df-60c38540d0ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:53.906884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6e5d2-7899-4400-b4df-60c38540d0ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:53.909656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6e5d2-7899-4400-b4df-60c38540d0ea None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:53.922741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 406/831] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:53.928883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca58e0-a335-4e68-b30a-51e33ddfbee8 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:54.044779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca58e0-a335-4e68-b30a-51e33ddfbee8 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:54.081022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e597f1-034e-4f31-9f72-166ef7ebdaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:54.081689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e597f1-034e-4f31-9f72-166ef7ebdaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:54.081975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e597f1-034e-4f31-9f72-166ef7ebdaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:54.082254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e597f1-034e-4f31-9f72-166ef7ebdaf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:54.083153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e597f1-034e-4f31-9f72-166ef7ebdaf3 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=232cfaefaf584ac59720713e02cf2c2d) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:54.083472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e597f1-034e-4f31-9f72-166ef7ebdaf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:54.085001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e597f1-034e-4f31-9f72-166ef7ebdaf3 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:54.091979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 426/832] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:53 2022] GET /identity/v3/role_assignments?user.id=232cfaefaf584ac59720713e02cf2c2d&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:54.233749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362147f-e84a-44f6-a7c0-677dd4624b36 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:54.234174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362147f-e84a-44f6-a7c0-677dd4624b36 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:54.234566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362147f-e84a-44f6-a7c0-677dd4624b36 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:54.236401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 407/833] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:54.242478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce92a6-e914-491c-8366-ad53e5ab66a1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:54.242803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce92a6-e914-491c-8366-ad53e5ab66a1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:54.243636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce92a6-e914-491c-8366-ad53e5ab66a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:54.263303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f9ce92a6-e914-491c-8366-ad53e5ab66a1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:54.264896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f9ce92a6-e914-491c-8366-ad53e5ab66a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:54.314397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:54.350909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 427/834] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:16:54.375608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362147f-e84a-44f6-a7c0-677dd4624b36 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:54.483145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362147f-e84a-44f6-a7c0-677dd4624b36 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:54.523680 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad902090-5b6d-4fd6-9ce3-942498893d05 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:54.523985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3e2ae-3ba7-40fc-9c45-eed76e7e27b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:54.524103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad902090-5b6d-4fd6-9ce3-942498893d05 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:54.524834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad902090-5b6d-4fd6-9ce3-942498893d05 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:54.525633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e2ae-3ba7-40fc-9c45-eed76e7e27b4 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:54.526155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e2ae-3ba7-40fc-9c45-eed76e7e27b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:54.526481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e2ae-3ba7-40fc-9c45-eed76e7e27b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:54.526599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 428/835] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:54.531785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1c5fb-c2f1-4127-9c08-38cfbafd34d9 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:54.532072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1c5fb-c2f1-4127-9c08-38cfbafd34d9 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:54.532353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1c5fb-c2f1-4127-9c08-38cfbafd34d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:54.537917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3e2ae-3ba7-40fc-9c45-eed76e7e27b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-85c3e2ae-3ba7-40fc-9c45-eed76e7e27b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:54.547056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:54.548052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:54.548940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 408/836] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:16:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:54.551327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-eac1c5fb-c2f1-4127-9c08-38cfbafd34d9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:54.551704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-eac1c5fb-c2f1-4127-9c08-38cfbafd34d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:54.557764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3e2ae-3ba7-40fc-9c45-eed76e7e27b4 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:54.584683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:54.637040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 429/837] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:54.660813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1c5fb-c2f1-4127-9c08-38cfbafd34d9 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:54.710846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3e2ae-3ba7-40fc-9c45-eed76e7e27b4 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:54.760583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077ac84-e5ca-4d5c-8d00-5142e0b7bc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:54.761902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077ac84-e5ca-4d5c-8d00-5142e0b7bc19 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:54.762742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077ac84-e5ca-4d5c-8d00-5142e0b7bc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:54.763167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077ac84-e5ca-4d5c-8d00-5142e0b7bc19 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:54.764311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077ac84-e5ca-4d5c-8d00-5142e0b7bc19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:54.765040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077ac84-e5ca-4d5c-8d00-5142e0b7bc19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:54.767307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077ac84-e5ca-4d5c-8d00-5142e0b7bc19 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:54.771620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1c5fb-c2f1-4127-9c08-38cfbafd34d9 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:54.775156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:54.781008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 409/838] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:16:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:54.786344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077ac84-e5ca-4d5c-8d00-5142e0b7bc19 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:54.817223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd4828-46dd-4300-98bf-1d42d794c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:54.819074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd4828-46dd-4300-98bf-1d42d794c9cf None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:54.819770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd4828-46dd-4300-98bf-1d42d794c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:54.822144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd4828-46dd-4300-98bf-1d42d794c9cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:54.836028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd4828-46dd-4300-98bf-1d42d794c9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-20fd4828-46dd-4300-98bf-1d42d794c9cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:54.847419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:54.848350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:54.849225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 430/839] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:54.857790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd4828-46dd-4300-98bf-1d42d794c9cf None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:54.889812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:54.911931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077ac84-e5ca-4d5c-8d00-5142e0b7bc19 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:54.948228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e18f96-6d8e-49f2-bf0d-13f4e42955e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:54.949434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e18f96-6d8e-49f2-bf0d-13f4e42955e6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:54.949981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e18f96-6d8e-49f2-bf0d-13f4e42955e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:54.950457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e18f96-6d8e-49f2-bf0d-13f4e42955e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:54.963235 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e18f96-6d8e-49f2-bf0d-13f4e42955e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:54.965119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e18f96-6d8e-49f2-bf0d-13f4e42955e6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:54.981271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd4828-46dd-4300-98bf-1d42d794c9cf None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-20e18f96-6d8e-49f2-bf0d-13f4e42955e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:54.984132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:54.985051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:54.985935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:54.985935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:54.985935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:54.985935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:54.992665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 410/840] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:54.993911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e18f96-6d8e-49f2-bf0d-13f4e42955e6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.017794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9ec1f-f15d-453d-86e4-35f94efc08cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:55.018470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9ec1f-f15d-453d-86e4-35f94efc08cd None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:55.018753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9ec1f-f15d-453d-86e4-35f94efc08cd None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:55.019027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9ec1f-f15d-453d-86e4-35f94efc08cd None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:55.019958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9ec1f-f15d-453d-86e4-35f94efc08cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:55.020069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9ec1f-f15d-453d-86e4-35f94efc08cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:55.021460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9ec1f-f15d-453d-86e4-35f94efc08cd None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:55.036309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 431/841] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:16:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:55.036801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9ec1f-f15d-453d-86e4-35f94efc08cd None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.066861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:55.112703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e18f96-6d8e-49f2-bf0d-13f4e42955e6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:55.145840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:55.151578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9ec1f-f15d-453d-86e4-35f94efc08cd None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:55.153060 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba69f3-58e0-47bf-9dfd-98a27520253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:55.154040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba69f3-58e0-47bf-9dfd-98a27520253a None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:55.154575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba69f3-58e0-47bf-9dfd-98a27520253a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:55.155050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba69f3-58e0-47bf-9dfd-98a27520253a None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:55.155976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba69f3-58e0-47bf-9dfd-98a27520253a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:55.156547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba69f3-58e0-47bf-9dfd-98a27520253a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:55.158240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba69f3-58e0-47bf-9dfd-98a27520253a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:55.176177 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 411/842] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:55.177397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba69f3-58e0-47bf-9dfd-98a27520253a None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.190271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910dc07-74cb-4d22-a049-68b664427009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:55.191858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910dc07-74cb-4d22-a049-68b664427009 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:55.192660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910dc07-74cb-4d22-a049-68b664427009 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:55.193183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910dc07-74cb-4d22-a049-68b664427009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:55.208599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910dc07-74cb-4d22-a049-68b664427009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:55.210716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910dc07-74cb-4d22-a049-68b664427009 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-4910dc07-74cb-4d22-a049-68b664427009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:55.233908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:55.234823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:55.235703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:55.235703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:55.235703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:55.235703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:55.238282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 432/843] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:55.244240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910dc07-74cb-4d22-a049-68b664427009 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.293953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba69f3-58e0-47bf-9dfd-98a27520253a None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:55.327081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:55.331743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99727309-33fe-421f-b8b5-4c6896be7077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:55.334075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99727309-33fe-421f-b8b5-4c6896be7077 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:55.334585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99727309-33fe-421f-b8b5-4c6896be7077 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:55.335058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99727309-33fe-421f-b8b5-4c6896be7077 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:55.345552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99727309-33fe-421f-b8b5-4c6896be7077 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:55.347970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99727309-33fe-421f-b8b5-4c6896be7077 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:55.354467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910dc07-74cb-4d22-a049-68b664427009 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-99727309-33fe-421f-b8b5-4c6896be7077 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:55.367561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:55.369190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:16:55.370109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:55.372945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 412/844] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:16:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:55.376412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99727309-33fe-421f-b8b5-4c6896be7077 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.406512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dcc42-8aff-4d8b-be5f-08ef16b3cb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:55.407753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dcc42-8aff-4d8b-be5f-08ef16b3cb5b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:55.408239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dcc42-8aff-4d8b-be5f-08ef16b3cb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:55.408737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dcc42-8aff-4d8b-be5f-08ef16b3cb5b None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:55.409714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9dcc42-8aff-4d8b-be5f-08ef16b3cb5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:55.410387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9dcc42-8aff-4d8b-be5f-08ef16b3cb5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:55.412126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9dcc42-8aff-4d8b-be5f-08ef16b3cb5b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:55.432473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 433/845] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:16:55.433738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dcc42-8aff-4d8b-be5f-08ef16b3cb5b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.439020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:55.489758 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99727309-33fe-421f-b8b5-4c6896be7077 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:55.527049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e515e91-9280-4386-89a9-56bce47dd57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:55.527722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e515e91-9280-4386-89a9-56bce47dd57e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:55.528005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e515e91-9280-4386-89a9-56bce47dd57e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:55.528281 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e515e91-9280-4386-89a9-56bce47dd57e None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:55.529441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e515e91-9280-4386-89a9-56bce47dd57e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:55.529551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e515e91-9280-4386-89a9-56bce47dd57e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:55.531021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e515e91-9280-4386-89a9-56bce47dd57e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:55.549942 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dcc42-8aff-4d8b-be5f-08ef16b3cb5b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:55.554664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 413/846] 10.222.0.30 () {56 vars in 1149 bytes} [Mon May 23 15:16:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:55.555643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e515e91-9280-4386-89a9-56bce47dd57e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.594204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b32fb-6ca7-4c99-bde9-c7beca125615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:55.596506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b32fb-6ca7-4c99-bde9-c7beca125615 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:55.597062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b32fb-6ca7-4c99-bde9-c7beca125615 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:55.597567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b32fb-6ca7-4c99-bde9-c7beca125615 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:55.611146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b32fb-6ca7-4c99-bde9-c7beca125615 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:55.613740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b32fb-6ca7-4c99-bde9-c7beca125615 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:55.616781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-2c3b32fb-6ca7-4c99-bde9-c7beca125615 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:16:55.638804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:55.639963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:16:55.641122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:55.649061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 434/847] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:16:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:55.650113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b32fb-6ca7-4c99-bde9-c7beca125615 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.677523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e515e91-9280-4386-89a9-56bce47dd57e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:55.690688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:55.718492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f62771-d0cf-4eda-bbdf-518e60f16a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:55.719307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f62771-d0cf-4eda-bbdf-518e60f16a84 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:55.719690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f62771-d0cf-4eda-bbdf-518e60f16a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:55.720067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f62771-d0cf-4eda-bbdf-518e60f16a84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:55.792623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b32fb-6ca7-4c99-bde9-c7beca125615 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:55.798071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f62771-d0cf-4eda-bbdf-518e60f16a84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:55.799670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f62771-d0cf-4eda-bbdf-518e60f16a84 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:55.838877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998515c-0b5e-448a-961a-3c832d2687d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:55.839806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998515c-0b5e-448a-961a-3c832d2687d9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:55.840254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998515c-0b5e-448a-961a-3c832d2687d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:55.840694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998515c-0b5e-448a-961a-3c832d2687d9 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:55.841715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998515c-0b5e-448a-961a-3c832d2687d9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:55.842167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998515c-0b5e-448a-961a-3c832d2687d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:55.844251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998515c-0b5e-448a-961a-3c832d2687d9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:55.869554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 435/848] 10.222.0.30 () {56 vars in 1135 bytes} [Mon May 23 15:16:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:55.876565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998515c-0b5e-448a-961a-3c832d2687d9 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.921324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 414/849] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:16:55.923054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f62771-d0cf-4eda-bbdf-518e60f16a84 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:55.935789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:55.945131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:56.011199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998515c-0b5e-448a-961a-3c832d2687d9 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.049428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f62771-d0cf-4eda-bbdf-518e60f16a84 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.058325 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bbd01-272b-443f-8bcf-f8b36b0c82dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.059534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bbd01-272b-443f-8bcf-f8b36b0c82dd None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.059824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bbd01-272b-443f-8bcf-f8b36b0c82dd None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.060105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bbd01-272b-443f-8bcf-f8b36b0c82dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.089273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf0c0c-6214-46d7-92e3-c34f529fdb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.090451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf0c0c-6214-46d7-92e3-c34f529fdb65 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.090939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf0c0c-6214-46d7-92e3-c34f529fdb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.091415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf0c0c-6214-46d7-92e3-c34f529fdb65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.103635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf0c0c-6214-46d7-92e3-c34f529fdb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:56.105615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf0c0c-6214-46d7-92e3-c34f529fdb65 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-76bf0c0c-6214-46d7-92e3-c34f529fdb65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:56.126155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:56.127166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:56.128056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:56.128056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:56.128056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:56.128056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:56.134908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 415/850] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:56.136206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf0c0c-6214-46d7-92e3-c34f529fdb65 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:56.146214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bbd01-272b-443f-8bcf-f8b36b0c82dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:56.147695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bbd01-272b-443f-8bcf-f8b36b0c82dd None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:56.202782 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:56.209266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:56.246671 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf0c0c-6214-46d7-92e3-c34f529fdb65 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.256441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 436/851] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:16:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:16:56.257624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8e338-816a-4207-a661-94898f05a99c None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.257912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8e338-816a-4207-a661-94898f05a99c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.258186 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8e338-816a-4207-a661-94898f05a99c None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.264067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 437/852] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:56.264067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8e338-816a-4207-a661-94898f05a99c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:56.289477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3b372-5982-4eb5-a38b-11eb828e474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.290747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3b372-5982-4eb5-a38b-11eb828e474e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.291040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3b372-5982-4eb5-a38b-11eb828e474e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.291302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3b372-5982-4eb5-a38b-11eb828e474e None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.292035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3b372-5982-4eb5-a38b-11eb828e474e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:56.292346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3b372-5982-4eb5-a38b-11eb828e474e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:56.293790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3b372-5982-4eb5-a38b-11eb828e474e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:56.312604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 416/853] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:56.312604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc0775-b7dd-4dea-92fa-07dde8e4062d None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.312604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc0775-b7dd-4dea-92fa-07dde8e4062d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.312604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc0775-b7dd-4dea-92fa-07dde8e4062d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.331028 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ddbc0775-b7dd-4dea-92fa-07dde8e4062d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:56.332496 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ddbc0775-b7dd-4dea-92fa-07dde8e4062d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:56.383388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8e338-816a-4207-a661-94898f05a99c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.419566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 417/854] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:56.421048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc0775-b7dd-4dea-92fa-07dde8e4062d None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:56.440949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf9bd9-ce4b-49b4-bebb-7dcce4000988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.442058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9bd9-ce4b-49b4-bebb-7dcce4000988 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.442601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9bd9-ce4b-49b4-bebb-7dcce4000988 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.443080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9bd9-ce4b-49b4-bebb-7dcce4000988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.458362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf9bd9-ce4b-49b4-bebb-7dcce4000988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:56.459875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf9bd9-ce4b-49b4-bebb-7dcce4000988 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:56.476989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-b4bf9bd9-ce4b-49b4-bebb-7dcce4000988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:16:56.482383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:56.483534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:56.484548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:16:56.484548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:16:56.484548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:16:56.484548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:56.484548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 438/855] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:56.485057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf9bd9-ce4b-49b4-bebb-7dcce4000988 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:56.497477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:56.535475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc0775-b7dd-4dea-92fa-07dde8e4062d None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.571098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394731-b483-4cac-aa9c-2d1cc15eed44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.571994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394731-b483-4cac-aa9c-2d1cc15eed44 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.572274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394731-b483-4cac-aa9c-2d1cc15eed44 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.572550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394731-b483-4cac-aa9c-2d1cc15eed44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.579397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394731-b483-4cac-aa9c-2d1cc15eed44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:56.580837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394731-b483-4cac-aa9c-2d1cc15eed44 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:56.591382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf9bd9-ce4b-49b4-bebb-7dcce4000988 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-58394731-b483-4cac-aa9c-2d1cc15eed44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:56.594928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:56.595960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:56.596889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:56.596889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:56.596889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:56.596889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:56.596889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:56.596889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:56.596889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:56.596889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:56.596889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 418/856] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:56.597409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394731-b483-4cac-aa9c-2d1cc15eed44 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:56.628343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91f2c1-6ca9-4996-acf0-4f0486a8d5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.629203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91f2c1-6ca9-4996-acf0-4f0486a8d5aa None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.629481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91f2c1-6ca9-4996-acf0-4f0486a8d5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.629747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91f2c1-6ca9-4996-acf0-4f0486a8d5aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.630391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91f2c1-6ca9-4996-acf0-4f0486a8d5aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:56.631731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91f2c1-6ca9-4996-acf0-4f0486a8d5aa None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-2b91f2c1-6ca9-4996-acf0-4f0486a8d5aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:56.643951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:56.645173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:56.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 439/857] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:16:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:56.646808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96f1e6-69a1-442a-bd79-6560dc3fdf46 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.646808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96f1e6-69a1-442a-bd79-6560dc3fdf46 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.647521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96f1e6-69a1-442a-bd79-6560dc3fdf46 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.648502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 440/858] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:56.649257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96f1e6-69a1-442a-bd79-6560dc3fdf46 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:56.702399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394731-b483-4cac-aa9c-2d1cc15eed44 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.715177 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:56.720364 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:56.737137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a482e-442e-4615-9078-d05a8d466230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.737803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a482e-442e-4615-9078-d05a8d466230 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.738083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a482e-442e-4615-9078-d05a8d466230 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.738359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a482e-442e-4615-9078-d05a8d466230 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.739090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a482e-442e-4615-9078-d05a8d466230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:56.739398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a482e-442e-4615-9078-d05a8d466230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:56.740783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a482e-442e-4615-9078-d05a8d466230 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:56.752095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96f1e6-69a1-442a-bd79-6560dc3fdf46 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.762452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 419/859] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:56.763722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85917220-616d-41a8-bb92-d30d5f695c97 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.764010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85917220-616d-41a8-bb92-d30d5f695c97 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.764312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85917220-616d-41a8-bb92-d30d5f695c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.782087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-85917220-616d-41a8-bb92-d30d5f695c97 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:56.783234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-85917220-616d-41a8-bb92-d30d5f695c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:56.793243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d687362-cadf-489f-8c23-bb21c14fae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.793915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d687362-cadf-489f-8c23-bb21c14fae91 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.794199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d687362-cadf-489f-8c23-bb21c14fae91 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.794475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d687362-cadf-489f-8c23-bb21c14fae91 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.795263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d687362-cadf-489f-8c23-bb21c14fae91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:56.795575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d687362-cadf-489f-8c23-bb21c14fae91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:56.797023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d687362-cadf-489f-8c23-bb21c14fae91 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:56.809567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 441/860] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:56.810372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d687362-cadf-489f-8c23-bb21c14fae91 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:56.854977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 420/861] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:56.856050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85917220-616d-41a8-bb92-d30d5f695c97 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:56.929072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d687362-cadf-489f-8c23-bb21c14fae91 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.953650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85917220-616d-41a8-bb92-d30d5f695c97 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:56.955178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:56.959086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:56.964649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366612c4-2942-4bde-a5af-bb53655696c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.965358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366612c4-2942-4bde-a5af-bb53655696c7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.965642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366612c4-2942-4bde-a5af-bb53655696c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.965919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366612c4-2942-4bde-a5af-bb53655696c7 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:56.966620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366612c4-2942-4bde-a5af-bb53655696c7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:56.966928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366612c4-2942-4bde-a5af-bb53655696c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:56.968288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366612c4-2942-4bde-a5af-bb53655696c7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:56.972749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 442/862] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:16:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:56.973560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366612c4-2942-4bde-a5af-bb53655696c7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:56.998121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773ae0d-c8e7-42e7-a686-8e21056286c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:56.999202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ae0d-c8e7-42e7-a686-8e21056286c8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:56.999482 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ae0d-c8e7-42e7-a686-8e21056286c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:56.999760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ae0d-c8e7-42e7-a686-8e21056286c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:57.007466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773ae0d-c8e7-42e7-a686-8e21056286c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:57.009044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773ae0d-c8e7-42e7-a686-8e21056286c8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-d773ae0d-c8e7-42e7-a686-8e21056286c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:16:57.022927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:57.023843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 421/863] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:16:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:57.025494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773ae0d-c8e7-42e7-a686-8e21056286c8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:57.082428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366612c4-2942-4bde-a5af-bb53655696c7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:57.124472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958164f-0947-410c-9789-0085cde9cc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:57.125225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958164f-0947-410c-9789-0085cde9cc7a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:57.125532 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958164f-0947-410c-9789-0085cde9cc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:57.125920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958164f-0947-410c-9789-0085cde9cc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:57.127246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958164f-0947-410c-9789-0085cde9cc7a None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=0bb6b4503a4e43f480af8430e7f4fc49) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:57.127589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958164f-0947-410c-9789-0085cde9cc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:57.130426 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958164f-0947-410c-9789-0085cde9cc7a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:57.134793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773ae0d-c8e7-42e7-a686-8e21056286c8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:57.138255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 443/864] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:56 2022] GET /identity/v3/role_assignments?user.id=0bb6b4503a4e43f480af8430e7f4fc49&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:57.139080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958164f-0947-410c-9789-0085cde9cc7a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:57.175492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f3f3a-eb31-412f-bbc1-70b14ecaf7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:57.176785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f3f3a-eb31-412f-bbc1-70b14ecaf7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:57.177397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f3f3a-eb31-412f-bbc1-70b14ecaf7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:57.177516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f3f3a-eb31-412f-bbc1-70b14ecaf7f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:57.178202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f3f3a-eb31-412f-bbc1-70b14ecaf7f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:57.179898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f3f3a-eb31-412f-bbc1-70b14ecaf7f8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-519f3f3a-eb31-412f-bbc1-70b14ecaf7f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:57.195751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:57.197035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:16:57.204070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 422/865] 10.222.0.30 () {56 vars in 1132 bytes} [Mon May 23 15:16:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:16:57.204950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f3f3a-eb31-412f-bbc1-70b14ecaf7f8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:57.234134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:57.241947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:57.297296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958164f-0947-410c-9789-0085cde9cc7a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:57.335849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f3f3a-eb31-412f-bbc1-70b14ecaf7f8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:57.350438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cb80-28f1-4c16-a838-2aa324d42b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:57.351560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cb80-28f1-4c16-a838-2aa324d42b61 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:57.351855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cb80-28f1-4c16-a838-2aa324d42b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:57.352139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cb80-28f1-4c16-a838-2aa324d42b61 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:57.353457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6cb80-28f1-4c16-a838-2aa324d42b61 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:57.353917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6cb80-28f1-4c16-a838-2aa324d42b61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:57.355881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6cb80-28f1-4c16-a838-2aa324d42b61 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:57.362029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 444/866] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:16:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:57.363208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cb80-28f1-4c16-a838-2aa324d42b61 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:57.377359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c1e1c-eb97-4199-b6ad-fbf85a85ab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:57.378209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c1e1c-eb97-4199-b6ad-fbf85a85ab20 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:57.378492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c1e1c-eb97-4199-b6ad-fbf85a85ab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:57.379181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c1e1c-eb97-4199-b6ad-fbf85a85ab20 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:57.379619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c1e1c-eb97-4199-b6ad-fbf85a85ab20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:57.379929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c1e1c-eb97-4199-b6ad-fbf85a85ab20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:57.381804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c1e1c-eb97-4199-b6ad-fbf85a85ab20 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:57.397861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 423/867] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:16:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:57.406377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c1e1c-eb97-4199-b6ad-fbf85a85ab20 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:57.526975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cb80-28f1-4c16-a838-2aa324d42b61 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:57.528392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:57.551429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c1e1c-eb97-4199-b6ad-fbf85a85ab20 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:57.559165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:57.574861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898a81a-4ea5-4960-911e-0c592f499cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:57.576173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898a81a-4ea5-4960-911e-0c592f499cbf None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:57.576173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898a81a-4ea5-4960-911e-0c592f499cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:57.576173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898a81a-4ea5-4960-911e-0c592f499cbf None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:57.576981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898a81a-4ea5-4960-911e-0c592f499cbf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:57.577290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898a81a-4ea5-4960-911e-0c592f499cbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:57.578658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898a81a-4ea5-4960-911e-0c592f499cbf None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:57.586521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 445/868] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:16:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:57.594526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898a81a-4ea5-4960-911e-0c592f499cbf None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:57.601607 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04aca5f-0281-45b3-b383-10fbe2356d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:57.602674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04aca5f-0281-45b3-b383-10fbe2356d7e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:57.603306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04aca5f-0281-45b3-b383-10fbe2356d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:57.603841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04aca5f-0281-45b3-b383-10fbe2356d7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:57.604926 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04aca5f-0281-45b3-b383-10fbe2356d7e None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=ff6c489d950a4370bbf01866d2fc8fc3) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:57.606042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04aca5f-0281-45b3-b383-10fbe2356d7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:57.608287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04aca5f-0281-45b3-b383-10fbe2356d7e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:57.620185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 424/869] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:16:57 2022] GET /identity/v3/role_assignments?user.id=ff6c489d950a4370bbf01866d2fc8fc3&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:57.718507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898a81a-4ea5-4960-911e-0c592f499cbf None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:57.759838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb4af9-1368-47e9-8dab-cfb5772ae343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:57.760579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb4af9-1368-47e9-8dab-cfb5772ae343 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:57.760861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb4af9-1368-47e9-8dab-cfb5772ae343 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:57.761172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb4af9-1368-47e9-8dab-cfb5772ae343 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:57.761896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb4af9-1368-47e9-8dab-cfb5772ae343 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:16:57.762263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb4af9-1368-47e9-8dab-cfb5772ae343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:57.763671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb4af9-1368-47e9-8dab-cfb5772ae343 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:16:57.770049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 446/870] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:16:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:16:59.578692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ea32c-10f0-45ab-9186-73221032d52f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:59.579214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ea32c-10f0-45ab-9186-73221032d52f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:59.579214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ea32c-10f0-45ab-9186-73221032d52f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:59.580898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 425/871] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:59.586450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7724922-ef6e-4e1e-ad79-4c33916cb9ca None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:59.587375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7724922-ef6e-4e1e-ad79-4c33916cb9ca None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:59.587570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7724922-ef6e-4e1e-ad79-4c33916cb9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:59.612888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d7724922-ef6e-4e1e-ad79-4c33916cb9ca None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:59.612888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d7724922-ef6e-4e1e-ad79-4c33916cb9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:59.661111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:59.678426 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3761e73-b26c-4571-9b9c-a6f0146719b1 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:59.678729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3761e73-b26c-4571-9b9c-a6f0146719b1 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:59.678986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3761e73-b26c-4571-9b9c-a6f0146719b1 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:59.682050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 426/872] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:16:59.683479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 447/873] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:16:59.688031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c410139-01f5-4675-9a2b-2471a38f7617 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:59.688313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c410139-01f5-4675-9a2b-2471a38f7617 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:59.688589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c410139-01f5-4675-9a2b-2471a38f7617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:59.705275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-0c410139-01f5-4675-9a2b-2471a38f7617 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:16:59.706446 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0c410139-01f5-4675-9a2b-2471a38f7617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:16:59.708995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7724922-ef6e-4e1e-ad79-4c33916cb9ca None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:59.796964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 427/874] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:16:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:16:59.818793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c410139-01f5-4675-9a2b-2471a38f7617 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:59.826993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7724922-ef6e-4e1e-ad79-4c33916cb9ca None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:59.863208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b77423-1e40-4037-b7fc-3cf73b08108f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:59.864036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b77423-1e40-4037-b7fc-3cf73b08108f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:59.864322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b77423-1e40-4037-b7fc-3cf73b08108f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:59.864596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b77423-1e40-4037-b7fc-3cf73b08108f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:59.876615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b77423-1e40-4037-b7fc-3cf73b08108f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-73b77423-1e40-4037-b7fc-3cf73b08108f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:16:59.886187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:16:59.887193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:16:59.888017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 448/875] 10.222.0.30 () {56 vars in 1120 bytes} [Mon May 23 15:16:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:16:59.896458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b77423-1e40-4037-b7fc-3cf73b08108f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:16:59.918615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:59.943283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c410139-01f5-4675-9a2b-2471a38f7617 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:16:59.945775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:16:59.988231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f38f7-672d-432b-8333-cd3ff76a05b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:16:59.989484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f38f7-672d-432b-8333-cd3ff76a05b6 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:16:59.990087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f38f7-672d-432b-8333-cd3ff76a05b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:16:59.990560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f38f7-672d-432b-8333-cd3ff76a05b6 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:16:59.991504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f38f7-672d-432b-8333-cd3ff76a05b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:16:59.994163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f38f7-672d-432b-8333-cd3ff76a05b6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:00.011092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 428/876] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:16:59 2022] POST /identity/v3/services => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:17:00.011998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b77423-1e40-4037-b7fc-3cf73b08108f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:00.065780 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5c078-7f8b-4b38-a380-b1e07bfea7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:00.066875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5c078-7f8b-4b38-a380-b1e07bfea7cf None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.067362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5c078-7f8b-4b38-a380-b1e07bfea7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.067862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5c078-7f8b-4b38-a380-b1e07bfea7cf None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.068840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5c078-7f8b-4b38-a380-b1e07bfea7cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:00.069399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5c078-7f8b-4b38-a380-b1e07bfea7cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:00.070995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5c078-7f8b-4b38-a380-b1e07bfea7cf None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:00.085308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 449/877] 10.222.0.30 () {56 vars in 1115 bytes} [Mon May 23 15:16:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:00.088303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f38f7-672d-432b-8333-cd3ff76a05b6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:00.108397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f4ca5-ea3f-4bb3-accb-0831a671311a None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.108781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f4ca5-ea3f-4bb3-accb-0831a671311a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.109095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f4ca5-ea3f-4bb3-accb-0831a671311a None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.112385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 450/878] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:17:00.118063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8863c-e934-49f7-ba72-5e3f95ea6955 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.118855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8863c-e934-49f7-ba72-5e3f95ea6955 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.119260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8863c-e934-49f7-ba72-5e3f95ea6955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.143009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1bb8863c-e934-49f7-ba72-5e3f95ea6955 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:00.144229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1bb8863c-e934-49f7-ba72-5e3f95ea6955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:00.220927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f38f7-672d-432b-8333-cd3ff76a05b6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:00.229815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 451/879] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:00.231355 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5e8b6-2d44-4106-b0d8-f513cd9d3402 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.231639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5e8b6-2d44-4106-b0d8-f513cd9d3402 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.231912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5e8b6-2d44-4106-b0d8-f513cd9d3402 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.237188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 452/880] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:00.238732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0c5ac-91b1-44a0-a402-c8580c2018d8 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.239017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0c5ac-91b1-44a0-a402-c8580c2018d8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.239284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0c5ac-91b1-44a0-a402-c8580c2018d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.259818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-42d0c5ac-91b1-44a0-a402-c8580c2018d8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:00.261425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-42d0c5ac-91b1-44a0-a402-c8580c2018d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:00.266853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ec60d-af6b-438e-bd05-45c148f2d94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:00.267650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ec60d-af6b-438e-bd05-45c148f2d94b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.267932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ec60d-af6b-438e-bd05-45c148f2d94b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.268204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ec60d-af6b-438e-bd05-45c148f2d94b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.278146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:00.280260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ec60d-af6b-438e-bd05-45c148f2d94b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:00.281743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ec60d-af6b-438e-bd05-45c148f2d94b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:00.291182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-d98ec60d-af6b-438e-bd05-45c148f2d94b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:00.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:00.306365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:00.307634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:00.307634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:00.307634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:00.307634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:00.307634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 429/881] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:00.308507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ec60d-af6b-438e-bd05-45c148f2d94b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:00.339217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 453/882] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:00.340294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0c5ac-91b1-44a0-a402-c8580c2018d8 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:00.433894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ec60d-af6b-438e-bd05-45c148f2d94b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:00.450276 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0c5ac-91b1-44a0-a402-c8580c2018d8 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:00.473565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebb0d9-f18e-4e07-9ca4-1bbe8c2f5512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:00.474406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebb0d9-f18e-4e07-9ca4-1bbe8c2f5512 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.474635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebb0d9-f18e-4e07-9ca4-1bbe8c2f5512 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.474903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebb0d9-f18e-4e07-9ca4-1bbe8c2f5512 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.475665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebb0d9-f18e-4e07-9ca4-1bbe8c2f5512 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:00.477158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebb0d9-f18e-4e07-9ca4-1bbe8c2f5512 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:00.487713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350f140-cc52-48e2-881f-9d558f928b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:00.488371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f140-cc52-48e2-881f-9d558f928b21 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.488654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f140-cc52-48e2-881f-9d558f928b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.488958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f140-cc52-48e2-881f-9d558f928b21 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.489675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350f140-cc52-48e2-881f-9d558f928b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:00.489983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350f140-cc52-48e2-881f-9d558f928b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-e6ebb0d9-f18e-4e07-9ca4-1bbe8c2f5512 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:00.491118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:00.492049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:00.497784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:00.498526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350f140-cc52-48e2-881f-9d558f928b21 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:00.498606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 430/883] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:17:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:00.498606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebb0d9-f18e-4e07-9ca4-1bbe8c2f5512 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:00.510349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 454/884] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:00.511269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350f140-cc52-48e2-881f-9d558f928b21 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:00.535959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:00.566760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:00.620855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebb0d9-f18e-4e07-9ca4-1bbe8c2f5512 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:00.624427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350f140-cc52-48e2-881f-9d558f928b21 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:00.658314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37986f-c6ce-4d05-a87d-0779feb9580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:00.658980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37986f-c6ce-4d05-a87d-0779feb9580c None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.659261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37986f-c6ce-4d05-a87d-0779feb9580c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.659533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37986f-c6ce-4d05-a87d-0779feb9580c None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.660251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37986f-c6ce-4d05-a87d-0779feb9580c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:00.661269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aedf29-2a3e-4d6e-93e5-52c6b0c9c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:00.661666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37986f-c6ce-4d05-a87d-0779feb9580c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:00.661935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aedf29-2a3e-4d6e-93e5-52c6b0c9c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.662213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aedf29-2a3e-4d6e-93e5-52c6b0c9c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.662486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aedf29-2a3e-4d6e-93e5-52c6b0c9c1a5 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.663190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aedf29-2a3e-4d6e-93e5-52c6b0c9c1a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:00.663494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aedf29-2a3e-4d6e-93e5-52c6b0c9c1a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:00.664964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aedf29-2a3e-4d6e-93e5-52c6b0c9c1a5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:00.676377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 455/885] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:00.685129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aedf29-2a3e-4d6e-93e5-52c6b0c9c1a5 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:00.685327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 431/886] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:17:00 2022] POST /identity/v3/services => generated 269 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:00.685327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37986f-c6ce-4d05-a87d-0779feb9580c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:00.812828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37986f-c6ce-4d05-a87d-0779feb9580c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:00.816719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:00.830058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:00.841155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aedf29-2a3e-4d6e-93e5-52c6b0c9c1a5 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:00.862100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d8a8d-226b-4f3d-8d34-2cc2ebb9516b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:00.862783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d8a8d-226b-4f3d-8d34-2cc2ebb9516b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.863117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d8a8d-226b-4f3d-8d34-2cc2ebb9516b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.863567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d8a8d-226b-4f3d-8d34-2cc2ebb9516b None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.864775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d8a8d-226b-4f3d-8d34-2cc2ebb9516b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:00.864775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d8a8d-226b-4f3d-8d34-2cc2ebb9516b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:00.866315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d8a8d-226b-4f3d-8d34-2cc2ebb9516b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:00.877057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 432/887] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:00.897313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89df42f-684c-4d83-952a-6b7888f772c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:00.898455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89df42f-684c-4d83-952a-6b7888f772c3 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:00.898898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89df42f-684c-4d83-952a-6b7888f772c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:00.899320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89df42f-684c-4d83-952a-6b7888f772c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:00.909683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89df42f-684c-4d83-952a-6b7888f772c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:00.911429 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89df42f-684c-4d83-952a-6b7888f772c3 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-c89df42f-684c-4d83-952a-6b7888f772c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:17:00.928995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:00.930194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:17:00.932021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:00.933813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 456/888] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:17:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:00.940648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d8a8d-226b-4f3d-8d34-2cc2ebb9516b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:01.058713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d8a8d-226b-4f3d-8d34-2cc2ebb9516b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:01.094516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd8b12-c53a-4704-81d0-9c1d5ab93fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:01.095196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd8b12-c53a-4704-81d0-9c1d5ab93fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:01.095477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd8b12-c53a-4704-81d0-9c1d5ab93fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:01.095751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd8b12-c53a-4704-81d0-9c1d5ab93fb4 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:01.096592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd8b12-c53a-4704-81d0-9c1d5ab93fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:01.096990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd8b12-c53a-4704-81d0-9c1d5ab93fb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:01.098486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd8b12-c53a-4704-81d0-9c1d5ab93fb4 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:01.117892 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 433/889] 10.222.0.30 () {56 vars in 1149 bytes} [Mon May 23 15:17:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:01.124151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89df42f-684c-4d83-952a-6b7888f772c3 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:01.240610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89df42f-684c-4d83-952a-6b7888f772c3 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:01.282117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0820-1eef-4201-9015-8f1c8bfd5018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:01.283210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0820-1eef-4201-9015-8f1c8bfd5018 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:01.283821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0820-1eef-4201-9015-8f1c8bfd5018 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:01.284337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0820-1eef-4201-9015-8f1c8bfd5018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:01.302518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:01.362210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c0820-1eef-4201-9015-8f1c8bfd5018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:01.364138 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c0820-1eef-4201-9015-8f1c8bfd5018 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:01.472040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 457/890] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:01 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:17:01.478930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd8b12-c53a-4704-81d0-9c1d5ab93fb4 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:01.517371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:01.595988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd8b12-c53a-4704-81d0-9c1d5ab93fb4 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:01.638505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0ffad-1028-40a5-8a3f-fcbfdc60a14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:01.639991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0ffad-1028-40a5-8a3f-fcbfdc60a14d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:01.640422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0ffad-1028-40a5-8a3f-fcbfdc60a14d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:01.640843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0ffad-1028-40a5-8a3f-fcbfdc60a14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:01.651755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0ffad-1028-40a5-8a3f-fcbfdc60a14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:01.653487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0ffad-1028-40a5-8a3f-fcbfdc60a14d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-41d0ffad-1028-40a5-8a3f-fcbfdc60a14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:01.672983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:01.674089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:01.675096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:01.675096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:01.675096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:01.675096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:01.676927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 434/891] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:01.688920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0820-1eef-4201-9015-8f1c8bfd5018 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:01.749508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:01.795907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0820-1eef-4201-9015-8f1c8bfd5018 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:01.831703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941dac-8ffe-41d9-86b9-9be86089e502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:01.832380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941dac-8ffe-41d9-86b9-9be86089e502 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:01.832662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941dac-8ffe-41d9-86b9-9be86089e502 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:01.832969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941dac-8ffe-41d9-86b9-9be86089e502 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:01.833691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e941dac-8ffe-41d9-86b9-9be86089e502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:01.834001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e941dac-8ffe-41d9-86b9-9be86089e502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:01.835357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e941dac-8ffe-41d9-86b9-9be86089e502 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:01.848241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 458/892] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:01.857043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0ffad-1028-40a5-8a3f-fcbfdc60a14d None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:01.971917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:01.990081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0ffad-1028-40a5-8a3f-fcbfdc60a14d None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:02.029681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d708a3-91c9-410c-ac67-0a91b1d288bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:02.030885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d708a3-91c9-410c-ac67-0a91b1d288bd None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:02.031377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d708a3-91c9-410c-ac67-0a91b1d288bd None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:02.031822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d708a3-91c9-410c-ac67-0a91b1d288bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:02.039467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d708a3-91c9-410c-ac67-0a91b1d288bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:02.041309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d708a3-91c9-410c-ac67-0a91b1d288bd None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-08d708a3-91c9-410c-ac67-0a91b1d288bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:17:02.056620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:02.057773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:02.058899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:17:02.058899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:17:02.058899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:17:02.058899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:17:02.058899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:17:02.058899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:17:02.058899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:02.058899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:02.065350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 435/893] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:02.067804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941dac-8ffe-41d9-86b9-9be86089e502 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:02.188487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941dac-8ffe-41d9-86b9-9be86089e502 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:02.190803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:02.227381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1d6ce-0136-4e97-babb-60589e083da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:02.228759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1d6ce-0136-4e97-babb-60589e083da9 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:02.229253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1d6ce-0136-4e97-babb-60589e083da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:02.229554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1d6ce-0136-4e97-babb-60589e083da9 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:02.230368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1d6ce-0136-4e97-babb-60589e083da9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:02.230682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1d6ce-0136-4e97-babb-60589e083da9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:02.232150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1d6ce-0136-4e97-babb-60589e083da9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:02.247471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 459/894] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:17:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:02.253814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d708a3-91c9-410c-ac67-0a91b1d288bd None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:02.364113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d708a3-91c9-410c-ac67-0a91b1d288bd None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:02.402458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53854f4f-2641-4d34-b317-fb5d1dc53984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:02.403571 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53854f4f-2641-4d34-b317-fb5d1dc53984 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:02.404090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53854f4f-2641-4d34-b317-fb5d1dc53984 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:02.404581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53854f4f-2641-4d34-b317-fb5d1dc53984 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:02.405715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53854f4f-2641-4d34-b317-fb5d1dc53984 None admin] RBAC: Adding query filter params (role.id=e3f9cda0e462469f92d47a2624887c97, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=0bb6b4503a4e43f480af8430e7f4fc49) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:02.406237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53854f4f-2641-4d34-b317-fb5d1dc53984 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:02.407424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d978f7a-6259-4310-be08-ea08486a8e39 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:02.407714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d978f7a-6259-4310-be08-ea08486a8e39 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:02.407990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d978f7a-6259-4310-be08-ea08486a8e39 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:02.408605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53854f4f-2641-4d34-b317-fb5d1dc53984 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:02.409605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 460/895] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:02.414572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceddb81-8633-414d-bb23-2725071f92c8 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:02.414858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceddb81-8633-414d-bb23-2725071f92c8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:02.415172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceddb81-8633-414d-bb23-2725071f92c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:02.416290 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 436/896] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:17:02 2022] GET /identity/v3/role_assignments?user.id=0bb6b4503a4e43f480af8430e7f4fc49&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=e3f9cda0e462469f92d47a2624887c97 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:02.436513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-3ceddb81-8633-414d-bb23-2725071f92c8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:02.437779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3ceddb81-8633-414d-bb23-2725071f92c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:02.518477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 461/897] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:02.542705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53854f4f-2641-4d34-b317-fb5d1dc53984 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:02.557380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:02.661328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53854f4f-2641-4d34-b317-fb5d1dc53984 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:02.697248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd2f9e-bebd-462f-b348-3a1629af86b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:02.698378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd2f9e-bebd-462f-b348-3a1629af86b8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:02.698657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd2f9e-bebd-462f-b348-3a1629af86b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:02.698932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd2f9e-bebd-462f-b348-3a1629af86b8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:02.699589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd2f9e-bebd-462f-b348-3a1629af86b8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:02.700992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd2f9e-bebd-462f-b348-3a1629af86b8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-bfdd2f9e-bebd-462f-b348-3a1629af86b8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:02.713690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:02.714696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:02.715522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 437/898] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:17:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:02.721093 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceddb81-8633-414d-bb23-2725071f92c8 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:02.834924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceddb81-8633-414d-bb23-2725071f92c8 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:02.848927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:02.874456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a98bb7-ddee-4f7f-8585-f3216c28869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:02.875126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a98bb7-ddee-4f7f-8585-f3216c28869f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:02.875407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a98bb7-ddee-4f7f-8585-f3216c28869f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:02.875683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a98bb7-ddee-4f7f-8585-f3216c28869f None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:02.876388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a98bb7-ddee-4f7f-8585-f3216c28869f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:02.876691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a98bb7-ddee-4f7f-8585-f3216c28869f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:02.878080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a98bb7-ddee-4f7f-8585-f3216c28869f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:02.883663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 462/899] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:17:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:02.889841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd2f9e-bebd-462f-b348-3a1629af86b8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:03.024136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:03.062771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd2f9e-bebd-462f-b348-3a1629af86b8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:03.120801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56792853-56eb-42ca-bb7c-bac97ac02815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:03.121902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56792853-56eb-42ca-bb7c-bac97ac02815 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.122358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56792853-56eb-42ca-bb7c-bac97ac02815 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.122801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56792853-56eb-42ca-bb7c-bac97ac02815 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.123690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56792853-56eb-42ca-bb7c-bac97ac02815 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:03.124498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56792853-56eb-42ca-bb7c-bac97ac02815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:03.128053 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56792853-56eb-42ca-bb7c-bac97ac02815 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:03.137503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 438/900] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:17:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:03.141795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b656dd-dda3-4005-8831-8912cc5efa6b None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.141795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b656dd-dda3-4005-8831-8912cc5efa6b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.141795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b656dd-dda3-4005-8831-8912cc5efa6b None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.142190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 463/901] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:03.145860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56792853-56eb-42ca-bb7c-bac97ac02815 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:03.147619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b2523-a90f-4dcc-a377-135c30d97499 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.148114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b2523-a90f-4dcc-a377-135c30d97499 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.148600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b2523-a90f-4dcc-a377-135c30d97499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.168540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6b1b2523-a90f-4dcc-a377-135c30d97499 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:03.171487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6b1b2523-a90f-4dcc-a377-135c30d97499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:03.294187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 464/902] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:03.296911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f4de9-87e2-4732-bdf4-6a74dab26e7b None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.297432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f4de9-87e2-4732-bdf4-6a74dab26e7b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.297432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f4de9-87e2-4732-bdf4-6a74dab26e7b None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.298976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 465/903] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:03.300732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56792853-56eb-42ca-bb7c-bac97ac02815 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:03.308370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a51523-e0da-4148-b212-6ddb13615102 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.309005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a51523-e0da-4148-b212-6ddb13615102 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.309457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a51523-e0da-4148-b212-6ddb13615102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.340477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d267a50-34df-47f3-b7cd-c45359e920c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:03.341309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d267a50-34df-47f3-b7cd-c45359e920c9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.341661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d267a50-34df-47f3-b7cd-c45359e920c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.342008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d267a50-34df-47f3-b7cd-c45359e920c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.342873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d267a50-34df-47f3-b7cd-c45359e920c9 None admin] RBAC: Adding query filter params (interface=public, service_id=336088dfa00547a598a523f7981f3ad3, region_id=RegionOne) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:03.343252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d267a50-34df-47f3-b7cd-c45359e920c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:03.343737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a5a51523-e0da-4148-b212-6ddb13615102 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:03.344756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d267a50-34df-47f3-b7cd-c45359e920c9 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:03.344938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a5a51523-e0da-4148-b212-6ddb13615102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:03.355386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 439/904] 10.222.0.30 () {56 vars in 1251 bytes} [Mon May 23 15:17:03 2022] GET /identity/v3/endpoints?service_id=336088dfa00547a598a523f7981f3ad3&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:03.355386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d267a50-34df-47f3-b7cd-c45359e920c9 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:03.358647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:03.440890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 466/905] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:03.440890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a51523-e0da-4148-b212-6ddb13615102 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:03.457233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:03.514802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d267a50-34df-47f3-b7cd-c45359e920c9 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:03.547381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a51523-e0da-4148-b212-6ddb13615102 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:03.553877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70b02f-4d00-45f3-a229-e39040dda2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:03.554767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70b02f-4d00-45f3-a229-e39040dda2cf None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.555119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70b02f-4d00-45f3-a229-e39040dda2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.555404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70b02f-4d00-45f3-a229-e39040dda2cf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.556719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70b02f-4d00-45f3-a229-e39040dda2cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:03.558122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70b02f-4d00-45f3-a229-e39040dda2cf None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-ca70b02f-4d00-45f3-a229-e39040dda2cf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:03.570978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:03.573625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:03.574901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 440/906] 10.222.0.30 () {56 vars in 1132 bytes} [Mon May 23 15:17:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:03.575795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70b02f-4d00-45f3-a229-e39040dda2cf None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:03.593701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58220936-a150-447b-b6ed-a1cadfc0e2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:03.594669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220936-a150-447b-b6ed-a1cadfc0e2eb None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.595031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220936-a150-447b-b6ed-a1cadfc0e2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.595376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220936-a150-447b-b6ed-a1cadfc0e2eb None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.596061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58220936-a150-447b-b6ed-a1cadfc0e2eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:03.597474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58220936-a150-447b-b6ed-a1cadfc0e2eb None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:03.615767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 467/907] 10.222.0.30 () {56 vars in 1087 bytes} [Mon May 23 15:17:03 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:17:03.615767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58220936-a150-447b-b6ed-a1cadfc0e2eb None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:03.680451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:03.711776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:03.736033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58220936-a150-447b-b6ed-a1cadfc0e2eb None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:03.740694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70b02f-4d00-45f3-a229-e39040dda2cf None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:03.776907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893447d6-5b37-4759-a29d-4bb134135aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:03.777908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893447d6-5b37-4759-a29d-4bb134135aff None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.778201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893447d6-5b37-4759-a29d-4bb134135aff None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.778482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893447d6-5b37-4759-a29d-4bb134135aff None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.779395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893447d6-5b37-4759-a29d-4bb134135aff None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:03.779711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893447d6-5b37-4759-a29d-4bb134135aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:03.781804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893447d6-5b37-4759-a29d-4bb134135aff None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:03.788490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 468/908] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:17:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:03.789550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27775660-5fb6-4a8f-9de7-2440a2447019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:03.790859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27775660-5fb6-4a8f-9de7-2440a2447019 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.791196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27775660-5fb6-4a8f-9de7-2440a2447019 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.791533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27775660-5fb6-4a8f-9de7-2440a2447019 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.792347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27775660-5fb6-4a8f-9de7-2440a2447019 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:03.794199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27775660-5fb6-4a8f-9de7-2440a2447019 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:03.795089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893447d6-5b37-4759-a29d-4bb134135aff None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:03.816715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 441/909] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:17:03 2022] POST /identity/v3/services => generated 241 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:03.940672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893447d6-5b37-4759-a29d-4bb134135aff None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:03.985612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230661a-c3dd-4a73-bf23-8be8d0c1139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:03.986300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230661a-c3dd-4a73-bf23-8be8d0c1139b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:03.986588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230661a-c3dd-4a73-bf23-8be8d0c1139b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:03.986930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230661a-c3dd-4a73-bf23-8be8d0c1139b None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:03.987654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230661a-c3dd-4a73-bf23-8be8d0c1139b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:03.987964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230661a-c3dd-4a73-bf23-8be8d0c1139b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:03.989390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230661a-c3dd-4a73-bf23-8be8d0c1139b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:03.996343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 469/910] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:17:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:04.002167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27775660-5fb6-4a8f-9de7-2440a2447019 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:04.114360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27775660-5fb6-4a8f-9de7-2440a2447019 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:04.142114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:04.153070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2486e0-ff88-465b-ad19-f0bb4b36c130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:04.153913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2486e0-ff88-465b-ad19-f0bb4b36c130 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:04.154254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2486e0-ff88-465b-ad19-f0bb4b36c130 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:04.154728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2486e0-ff88-465b-ad19-f0bb4b36c130 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:04.155519 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2486e0-ff88-465b-ad19-f0bb4b36c130 None admin] RBAC: Adding query filter params (interface=public, service_id=2121e205d6954a33a380ca27d0117e32, region_id=RegionOne) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:04.155833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2486e0-ff88-465b-ad19-f0bb4b36c130 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:04.157248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2486e0-ff88-465b-ad19-f0bb4b36c130 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:04.168128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 442/911] 10.222.0.30 () {56 vars in 1251 bytes} [Mon May 23 15:17:04 2022] GET /identity/v3/endpoints?service_id=2121e205d6954a33a380ca27d0117e32&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:04.169981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230661a-c3dd-4a73-bf23-8be8d0c1139b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:04.179408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:04.281395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230661a-c3dd-4a73-bf23-8be8d0c1139b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:04.316218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58e5ea-c5e4-4a20-a8db-65a417ab06d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:04.316904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58e5ea-c5e4-4a20-a8db-65a417ab06d3 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:04.317192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58e5ea-c5e4-4a20-a8db-65a417ab06d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:04.317463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58e5ea-c5e4-4a20-a8db-65a417ab06d3 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:04.318139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58e5ea-c5e4-4a20-a8db-65a417ab06d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:04.319490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58e5ea-c5e4-4a20-a8db-65a417ab06d3 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:04.326074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 470/912] 10.222.0.30 () {56 vars in 1087 bytes} [Mon May 23 15:17:04 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:17:04.809401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8ece7-f1a8-47d1-99e7-edc544a3b52c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:04.810409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8ece7-f1a8-47d1-99e7-edc544a3b52c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:04.810924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8ece7-f1a8-47d1-99e7-edc544a3b52c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:04.812741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 443/913] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:04.818370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f166c-723f-49df-9501-e0f327b25484 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:04.819186 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f166c-723f-49df-9501-e0f327b25484 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:04.819666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f166c-723f-49df-9501-e0f327b25484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:04.838044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-7c7f166c-723f-49df-9501-e0f327b25484 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:04.839739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7c7f166c-723f-49df-9501-e0f327b25484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:04.904857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:04.909960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 471/914] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:04.932712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8ece7-f1a8-47d1-99e7-edc544a3b52c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:05.041739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8ece7-f1a8-47d1-99e7-edc544a3b52c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:05.077081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e110ed6-f386-45c6-b261-43674523a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:05.077899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e110ed6-f386-45c6-b261-43674523a9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:05.078181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e110ed6-f386-45c6-b261-43674523a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:05.078457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e110ed6-f386-45c6-b261-43674523a9b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:05.089620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e110ed6-f386-45c6-b261-43674523a9b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-3e110ed6-f386-45c6-b261-43674523a9b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:05.097852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:05.098787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:05.099612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 444/915] 10.222.0.30 () {56 vars in 1120 bytes} [Mon May 23 15:17:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:05.106859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f166c-723f-49df-9501-e0f327b25484 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:05.209597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f166c-723f-49df-9501-e0f327b25484 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:05.246592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de440a1-9068-43ba-80ed-0159e5b27846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:05.247325 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de440a1-9068-43ba-80ed-0159e5b27846 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:05.247607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de440a1-9068-43ba-80ed-0159e5b27846 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:05.247880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de440a1-9068-43ba-80ed-0159e5b27846 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:05.248627 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de440a1-9068-43ba-80ed-0159e5b27846 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:05.248956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de440a1-9068-43ba-80ed-0159e5b27846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:05.250566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de440a1-9068-43ba-80ed-0159e5b27846 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:05.259814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 472/916] 10.222.0.30 () {56 vars in 1115 bytes} [Mon May 23 15:17:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:05.265740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e110ed6-f386-45c6-b261-43674523a9b9 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:05.315910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:05.377765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e110ed6-f386-45c6-b261-43674523a9b9 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:05.414214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76acdbf-511f-483f-8d99-b993ab353314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:05.415421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76acdbf-511f-483f-8d99-b993ab353314 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:05.415965 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76acdbf-511f-483f-8d99-b993ab353314 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:05.416463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76acdbf-511f-483f-8d99-b993ab353314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:05.427305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76acdbf-511f-483f-8d99-b993ab353314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:05.429033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76acdbf-511f-483f-8d99-b993ab353314 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-d76acdbf-511f-483f-8d99-b993ab353314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:05.447979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:05.448904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:05.449793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:05.449793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:05.449793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:05.449793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:05.452015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 445/917] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:05.457800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de440a1-9068-43ba-80ed-0159e5b27846 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:05.522610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:05.566436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de440a1-9068-43ba-80ed-0159e5b27846 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:05.612650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c15e67-6d21-4b60-be65-67e96cebfa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:05.613365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c15e67-6d21-4b60-be65-67e96cebfa9b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:05.613653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c15e67-6d21-4b60-be65-67e96cebfa9b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:05.613929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c15e67-6d21-4b60-be65-67e96cebfa9b None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:05.614656 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c15e67-6d21-4b60-be65-67e96cebfa9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:05.614959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c15e67-6d21-4b60-be65-67e96cebfa9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:05.616320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c15e67-6d21-4b60-be65-67e96cebfa9b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:05.630228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 473/918] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:05.636073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76acdbf-511f-483f-8d99-b993ab353314 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:05.753142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76acdbf-511f-483f-8d99-b993ab353314 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:05.755005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:05.791518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442c4d1-b72c-4fd7-a2cf-1e4f4c0ea3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:05.792744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442c4d1-b72c-4fd7-a2cf-1e4f4c0ea3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:05.793278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442c4d1-b72c-4fd7-a2cf-1e4f4c0ea3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:05.793750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442c4d1-b72c-4fd7-a2cf-1e4f4c0ea3c9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:05.801742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442c4d1-b72c-4fd7-a2cf-1e4f4c0ea3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:05.803459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442c4d1-b72c-4fd7-a2cf-1e4f4c0ea3c9 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-e442c4d1-b72c-4fd7-a2cf-1e4f4c0ea3c9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:17:05.819090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:05.820448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:17:05.821401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:05.824167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 446/919] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:17:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:05.830073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c15e67-6d21-4b60-be65-67e96cebfa9b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:05.932571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c15e67-6d21-4b60-be65-67e96cebfa9b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:05.962064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:05.976237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4011f-9997-404e-85c9-d4c5f9521fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:05.976959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4011f-9997-404e-85c9-d4c5f9521fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:05.977246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4011f-9997-404e-85c9-d4c5f9521fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:05.977521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4011f-9997-404e-85c9-d4c5f9521fb5 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:05.978517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4011f-9997-404e-85c9-d4c5f9521fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:05.978830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4011f-9997-404e-85c9-d4c5f9521fb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:05.980274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4011f-9997-404e-85c9-d4c5f9521fb5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:05.999048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 474/920] 10.222.0.30 () {56 vars in 1149 bytes} [Mon May 23 15:17:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:06.004916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442c4d1-b72c-4fd7-a2cf-1e4f4c0ea3c9 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.110153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442c4d1-b72c-4fd7-a2cf-1e4f4c0ea3c9 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:06.140324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d25539-a3c3-4f80-ba1b-d058c9e53659 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.140619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d25539-a3c3-4f80-ba1b-d058c9e53659 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.141000 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d25539-a3c3-4f80-ba1b-d058c9e53659 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.143078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 475/921] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:06.148757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240c63e-8a5b-42a7-a8cc-47dcedd4d766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:06.149589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240c63e-8a5b-42a7-a8cc-47dcedd4d766 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.150521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240c63e-8a5b-42a7-a8cc-47dcedd4d766 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.150725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d6c28-3686-402b-a220-4b10b7138414 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.150827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240c63e-8a5b-42a7-a8cc-47dcedd4d766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.151312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d6c28-3686-402b-a220-4b10b7138414 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.151788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d6c28-3686-402b-a220-4b10b7138414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.162987 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240c63e-8a5b-42a7-a8cc-47dcedd4d766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:06.164440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240c63e-8a5b-42a7-a8cc-47dcedd4d766 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:06.172475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-836d6c28-3686-402b-a220-4b10b7138414 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:06.174354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-836d6c28-3686-402b-a220-4b10b7138414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-5240c63e-8a5b-42a7-a8cc-47dcedd4d766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:06.182119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:06.183017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:06.183908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:06.183908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:06.183908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:06.183908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:06.183908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 447/922] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:06.188905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cb9ec-d588-4f05-a672-f9fe3c446245 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.188905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cb9ec-d588-4f05-a672-f9fe3c446245 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.188905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cb9ec-d588-4f05-a672-f9fe3c446245 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.188905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 448/923] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:06.188905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cb9ec-d588-4f05-a672-f9fe3c446245 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.206584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:06.256080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 476/924] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:06.257854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b2ac8-bbe2-444a-bdbe-2703d03f0791 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.258369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b2ac8-bbe2-444a-bdbe-2703d03f0791 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.258946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b2ac8-bbe2-444a-bdbe-2703d03f0791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.286222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b03b2ac8-bbe2-444a-bdbe-2703d03f0791 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:06.287510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b03b2ac8-bbe2-444a-bdbe-2703d03f0791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:06.301107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cb9ec-d588-4f05-a672-f9fe3c446245 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:06.340074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21063cab-6cd2-457d-9d61-33c23067a102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:06.340736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21063cab-6cd2-457d-9d61-33c23067a102 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.341050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21063cab-6cd2-457d-9d61-33c23067a102 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.341319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21063cab-6cd2-457d-9d61-33c23067a102 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.342134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21063cab-6cd2-457d-9d61-33c23067a102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:06.342439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21063cab-6cd2-457d-9d61-33c23067a102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:06.343801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21063cab-6cd2-457d-9d61-33c23067a102 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:06.360145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 449/925] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:06.360145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21063cab-6cd2-457d-9d61-33c23067a102 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.362990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 477/926] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:06.365808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b2ac8-bbe2-444a-bdbe-2703d03f0791 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.420470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:06.440805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:06.467307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21063cab-6cd2-457d-9d61-33c23067a102 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:06.487201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b2ac8-bbe2-444a-bdbe-2703d03f0791 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:06.510218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d6412-4858-4e58-9fdf-99fa6c041f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:06.511087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d6412-4858-4e58-9fdf-99fa6c041f17 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.511406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d6412-4858-4e58-9fdf-99fa6c041f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.511711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d6412-4858-4e58-9fdf-99fa6c041f17 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.512421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d6412-4858-4e58-9fdf-99fa6c041f17 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:06.513957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d6412-4858-4e58-9fdf-99fa6c041f17 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-6f7d6412-4858-4e58-9fdf-99fa6c041f17 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:06.526868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:06.527838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:06.534784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 450/927] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:17:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:06.535913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d6412-4858-4e58-9fdf-99fa6c041f17 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.535993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dc872-3782-49c8-af87-2f466b7e454d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:06.535993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dc872-3782-49c8-af87-2f466b7e454d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.535993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dc872-3782-49c8-af87-2f466b7e454d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.535993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dc872-3782-49c8-af87-2f466b7e454d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.544091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dc872-3782-49c8-af87-2f466b7e454d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:06.545698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dc872-3782-49c8-af87-2f466b7e454d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-3d6dc872-3782-49c8-af87-2f466b7e454d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:17:06.569529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:06.570480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:06.571359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:17:06.571359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:17:06.571359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:17:06.571359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:17:06.571359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:17:06.571359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:17:06.571359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:06.571359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:06.571359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 478/928] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:06.572546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dc872-3782-49c8-af87-2f466b7e454d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.656603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d6412-4858-4e58-9fdf-99fa6c041f17 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:06.680011 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:06.683825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dc872-3782-49c8-af87-2f466b7e454d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:06.692807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20c69f-2611-495c-ae56-6daa7baeb331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:06.693628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20c69f-2611-495c-ae56-6daa7baeb331 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.693911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20c69f-2611-495c-ae56-6daa7baeb331 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.694184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20c69f-2611-495c-ae56-6daa7baeb331 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.694840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20c69f-2611-495c-ae56-6daa7baeb331 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:06.696216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20c69f-2611-495c-ae56-6daa7baeb331 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-2e20c69f-2611-495c-ae56-6daa7baeb331 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:06.707970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:06.708945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:06.715095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 451/929] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:17:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:06.715867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20c69f-2611-495c-ae56-6daa7baeb331 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.716578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:06.718883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cca743-dd2b-44d0-a3d6-f528940bff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:06.719556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cca743-dd2b-44d0-a3d6-f528940bff60 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.719887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cca743-dd2b-44d0-a3d6-f528940bff60 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.720123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cca743-dd2b-44d0-a3d6-f528940bff60 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.720833 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cca743-dd2b-44d0-a3d6-f528940bff60 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:06.721167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cca743-dd2b-44d0-a3d6-f528940bff60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:06.722524 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cca743-dd2b-44d0-a3d6-f528940bff60 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:06.727496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 479/930] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:06.729304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27a84e-a240-4889-945c-324e99c16c09 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.729709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27a84e-a240-4889-945c-324e99c16c09 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.729878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27a84e-a240-4889-945c-324e99c16c09 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.731167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 480/931] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:06.731918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27a84e-a240-4889-945c-324e99c16c09 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.820279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20c69f-2611-495c-ae56-6daa7baeb331 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:06.836978 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27a84e-a240-4889-945c-324e99c16c09 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:06.852699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8a051-afe7-4b0f-838c-3cda043c6e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:06.854234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8a051-afe7-4b0f-838c-3cda043c6e41 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.854716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8a051-afe7-4b0f-838c-3cda043c6e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.855218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8a051-afe7-4b0f-838c-3cda043c6e41 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.856247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8a051-afe7-4b0f-838c-3cda043c6e41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:06.856554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8a051-afe7-4b0f-838c-3cda043c6e41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:06.858032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8a051-afe7-4b0f-838c-3cda043c6e41 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:06.871167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 452/932] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:17:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:06.872027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8a051-afe7-4b0f-838c-3cda043c6e41 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.873280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21fc6-a97b-44d3-b69b-1ca10901e965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:06.873961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21fc6-a97b-44d3-b69b-1ca10901e965 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.874496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21fc6-a97b-44d3-b69b-1ca10901e965 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.874777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21fc6-a97b-44d3-b69b-1ca10901e965 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.875499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e21fc6-a97b-44d3-b69b-1ca10901e965 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:06.875804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e21fc6-a97b-44d3-b69b-1ca10901e965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:06.877261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e21fc6-a97b-44d3-b69b-1ca10901e965 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:06.886183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 481/933] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:17:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:06.890257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60f143-abc6-48dc-a6da-2cc59536a822 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:06.890506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60f143-abc6-48dc-a6da-2cc59536a822 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:06.890732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60f143-abc6-48dc-a6da-2cc59536a822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:06.911112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1d60f143-abc6-48dc-a6da-2cc59536a822 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:06.912195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1d60f143-abc6-48dc-a6da-2cc59536a822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:06.936192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:06.944820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:06.980289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 482/934] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:06.981190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60f143-abc6-48dc-a6da-2cc59536a822 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:06.985439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8a051-afe7-4b0f-838c-3cda043c6e41 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.020267 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b1c4c-6532-4c8e-a85e-43c012d3253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.020963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b1c4c-6532-4c8e-a85e-43c012d3253f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.021245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b1c4c-6532-4c8e-a85e-43c012d3253f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.021516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b1c4c-6532-4c8e-a85e-43c012d3253f None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.022215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b1c4c-6532-4c8e-a85e-43c012d3253f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:07.022517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b1c4c-6532-4c8e-a85e-43c012d3253f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.023881 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b1c4c-6532-4c8e-a85e-43c012d3253f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.028835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 453/935] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:07.029806 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b1c4c-6532-4c8e-a85e-43c012d3253f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:07.080220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60f143-abc6-48dc-a6da-2cc59536a822 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.114806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bbce2-5756-45b7-a1d8-a56de1804b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.115683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bbce2-5756-45b7-a1d8-a56de1804b77 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.115960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bbce2-5756-45b7-a1d8-a56de1804b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.116229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bbce2-5756-45b7-a1d8-a56de1804b77 None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/0bb6b4503a4e43f480af8430e7f4fc49/roles/e3f9cda0e462469f92d47a2624887c97` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.118417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bbce2-5756-45b7-a1d8-a56de1804b77 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=0bb6b4503a4e43f480af8430e7f4fc49, role_id=e3f9cda0e462469f92d47a2624887c97)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.119884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bbce2-5756-45b7-a1d8-a56de1804b77 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.128797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b1c4c-6532-4c8e-a85e-43c012d3253f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.133318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 483/936] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:17:06 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/0bb6b4503a4e43f480af8430e7f4fc49/roles/e3f9cda0e462469f92d47a2624887c97 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:17:07.134300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bbce2-5756-45b7-a1d8-a56de1804b77 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:07.156957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:07.168099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9c89-e724-4756-88e4-1dd9aac451c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.168706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9c89-e724-4756-88e4-1dd9aac451c6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.169027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9c89-e724-4756-88e4-1dd9aac451c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.169308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9c89-e724-4756-88e4-1dd9aac451c6 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.170036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad9c89-e724-4756-88e4-1dd9aac451c6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:07.170348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad9c89-e724-4756-88e4-1dd9aac451c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.170572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:07.172277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad9c89-e724-4756-88e4-1dd9aac451c6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.178255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 454/937] 10.222.0.30 () {56 vars in 1114 bytes} [Mon May 23 15:17:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:07.182194 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9c89-e724-4756-88e4-1dd9aac451c6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:07.268964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bbce2-5756-45b7-a1d8-a56de1804b77 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.296445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9c89-e724-4756-88e4-1dd9aac451c6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.309851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1b75d-6996-4db1-adc6-981e7e0f55e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.310824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1b75d-6996-4db1-adc6-981e7e0f55e7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.311109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1b75d-6996-4db1-adc6-981e7e0f55e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.311429 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1b75d-6996-4db1-adc6-981e7e0f55e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.312218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1b75d-6996-4db1-adc6-981e7e0f55e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.314348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1b75d-6996-4db1-adc6-981e7e0f55e7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-11d1b75d-6996-4db1-adc6-981e7e0f55e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:07.328839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:07.329877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:17:07.330705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:07.332031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 484/938] 10.222.0.30 () {56 vars in 1132 bytes} [Mon May 23 15:17:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:07.332031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1b75d-6996-4db1-adc6-981e7e0f55e7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:07.343538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d8b63-11f9-48e3-8652-4675097f5009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.344614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d8b63-11f9-48e3-8652-4675097f5009 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.345115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d8b63-11f9-48e3-8652-4675097f5009 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.345391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d8b63-11f9-48e3-8652-4675097f5009 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.346381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d8b63-11f9-48e3-8652-4675097f5009 None admin] RBAC: Adding query filter params (interface=public, service_id=f75712e66aa14c8eb8b997ad6985dc2f, region_id=RegionOne) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:07.346698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d8b63-11f9-48e3-8652-4675097f5009 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.348536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d8b63-11f9-48e3-8652-4675097f5009 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.355673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 455/939] 10.222.0.30 () {56 vars in 1251 bytes} [Mon May 23 15:17:07 2022] GET /identity/v3/endpoints?service_id=f75712e66aa14c8eb8b997ad6985dc2f&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:07.356556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d8b63-11f9-48e3-8652-4675097f5009 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:07.420062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:07.430523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:07.450815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1b75d-6996-4db1-adc6-981e7e0f55e7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.471263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d8b63-11f9-48e3-8652-4675097f5009 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.486527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14eaed-e663-4ab0-84d1-56fbab7787a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.487194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14eaed-e663-4ab0-84d1-56fbab7787a9 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.487468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14eaed-e663-4ab0-84d1-56fbab7787a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.487737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14eaed-e663-4ab0-84d1-56fbab7787a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.488474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14eaed-e663-4ab0-84d1-56fbab7787a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.489869 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14eaed-e663-4ab0-84d1-56fbab7787a9 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.508461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 485/940] 10.222.0.30 () {60 vars in 1144 bytes} [Mon May 23 15:17:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:07.509842 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14eaed-e663-4ab0-84d1-56fbab7787a9 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:07.523906 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096265a-700d-4848-8981-ae62d98601cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.525537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096265a-700d-4848-8981-ae62d98601cd None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.526122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096265a-700d-4848-8981-ae62d98601cd None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.526603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096265a-700d-4848-8981-ae62d98601cd None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.527617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096265a-700d-4848-8981-ae62d98601cd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:07.528158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096265a-700d-4848-8981-ae62d98601cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.529640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096265a-700d-4848-8981-ae62d98601cd None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.540295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 456/941] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:17:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:07.541440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096265a-700d-4848-8981-ae62d98601cd None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:07.663976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14eaed-e663-4ab0-84d1-56fbab7787a9 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.677355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096265a-700d-4848-8981-ae62d98601cd None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.700195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:07.712815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f21bd8-63b1-48fe-82ec-78e0a2791852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.713684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f21bd8-63b1-48fe-82ec-78e0a2791852 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.715901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f21bd8-63b1-48fe-82ec-78e0a2791852 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.715901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f21bd8-63b1-48fe-82ec-78e0a2791852 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.716648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f21bd8-63b1-48fe-82ec-78e0a2791852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.718554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f21bd8-63b1-48fe-82ec-78e0a2791852 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.733605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 486/942] 10.222.0.30 () {56 vars in 1087 bytes} [Mon May 23 15:17:07 2022] GET /identity/v3/services => generated 1497 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:17:07.733796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0374d76-bda7-442c-81bb-338828bbdaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.733796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0374d76-bda7-442c-81bb-338828bbdaed None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.733796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0374d76-bda7-442c-81bb-338828bbdaed None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.733796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0374d76-bda7-442c-81bb-338828bbdaed None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.734659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0374d76-bda7-442c-81bb-338828bbdaed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:07.734759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0374d76-bda7-442c-81bb-338828bbdaed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.736092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0374d76-bda7-442c-81bb-338828bbdaed None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.742796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 457/943] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:17:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:07.749539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f21bd8-63b1-48fe-82ec-78e0a2791852 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:07.752996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:07.868443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f21bd8-63b1-48fe-82ec-78e0a2791852 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:07.907389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd4e20-89dc-476a-9f81-5bf9a1ba52aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:07.908352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd4e20-89dc-476a-9f81-5bf9a1ba52aa None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:07.908779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd4e20-89dc-476a-9f81-5bf9a1ba52aa None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:07.909237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd4e20-89dc-476a-9f81-5bf9a1ba52aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:07.910158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd4e20-89dc-476a-9f81-5bf9a1ba52aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:07.911711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd4e20-89dc-476a-9f81-5bf9a1ba52aa None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:07.934365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 487/944] 10.222.0.30 () {60 vars in 1144 bytes} [Mon May 23 15:17:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:09.431342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1631800-bac3-4baa-af03-bb0ca28cbc67 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:09.431894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1631800-bac3-4baa-af03-bb0ca28cbc67 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:09.432116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1631800-bac3-4baa-af03-bb0ca28cbc67 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:09.433719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 458/945] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:09.439066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f140c7a-6483-4fad-b0f5-e325a3e39591 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:09.439872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f140c7a-6483-4fad-b0f5-e325a3e39591 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:09.440389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f140c7a-6483-4fad-b0f5-e325a3e39591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:09.457906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0f140c7a-6483-4fad-b0f5-e325a3e39591 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:09.459392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0f140c7a-6483-4fad-b0f5-e325a3e39591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:09.516322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:09.539846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 488/946] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:09.564148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1631800-bac3-4baa-af03-bb0ca28cbc67 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:09.673760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1631800-bac3-4baa-af03-bb0ca28cbc67 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:09.709608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b68a2d-79c8-4799-abc4-afec91f07218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:09.710335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b68a2d-79c8-4799-abc4-afec91f07218 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:09.710612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b68a2d-79c8-4799-abc4-afec91f07218 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:09.710884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b68a2d-79c8-4799-abc4-afec91f07218 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:09.721752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b68a2d-79c8-4799-abc4-afec91f07218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-f4b68a2d-79c8-4799-abc4-afec91f07218 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:09.730631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:09.731544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:09.732364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 459/947] 10.222.0.30 () {56 vars in 1120 bytes} [Mon May 23 15:17:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:09.739431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f140c7a-6483-4fad-b0f5-e325a3e39591 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:09.851317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f140c7a-6483-4fad-b0f5-e325a3e39591 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:09.892826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0540dd-8806-4301-961f-8142bcaa76c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:09.893539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0540dd-8806-4301-961f-8142bcaa76c6 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:09.893822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0540dd-8806-4301-961f-8142bcaa76c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:09.894097 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0540dd-8806-4301-961f-8142bcaa76c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:09.894966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0540dd-8806-4301-961f-8142bcaa76c6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:09.895277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0540dd-8806-4301-961f-8142bcaa76c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:09.896649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0540dd-8806-4301-961f-8142bcaa76c6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:09.906147 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 489/948] 10.222.0.30 () {56 vars in 1115 bytes} [Mon May 23 15:17:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:09.911665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b68a2d-79c8-4799-abc4-afec91f07218 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:09.954742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:10.019664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b68a2d-79c8-4799-abc4-afec91f07218 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:10.032928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07e7bf-a57e-46ec-b7ae-1cd375725ca9 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.033265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07e7bf-a57e-46ec-b7ae-1cd375725ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.033542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07e7bf-a57e-46ec-b7ae-1cd375725ca9 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.035179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 490/949] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:10.041037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39ff8-664d-481a-80f5-015586c0759a None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.041373 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39ff8-664d-481a-80f5-015586c0759a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.041680 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39ff8-664d-481a-80f5-015586c0759a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.060893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30913b9a-34c6-4396-8962-78a7ace4e746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:10.061813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30913b9a-34c6-4396-8962-78a7ace4e746 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.062093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30913b9a-34c6-4396-8962-78a7ace4e746 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.062302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-05b39ff8-664d-481a-80f5-015586c0759a None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:10.062413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30913b9a-34c6-4396-8962-78a7ace4e746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.063719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-05b39ff8-664d-481a-80f5-015586c0759a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:10.077155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30913b9a-34c6-4396-8962-78a7ace4e746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:10.078825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30913b9a-34c6-4396-8962-78a7ace4e746 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-30913b9a-34c6-4396-8962-78a7ace4e746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:10.099305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:10.100255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:10.101155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:10.101155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:10.101155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:10.101155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:10.101155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 460/950] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:10.101955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906c265-f594-4c25-a4e6-490d6125bae2 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.102259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906c265-f594-4c25-a4e6-490d6125bae2 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.102535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906c265-f594-4c25-a4e6-490d6125bae2 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.104710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 461/951] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:10.107418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906c265-f594-4c25-a4e6-490d6125bae2 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.123987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:10.151492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 491/952] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:10.158752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c66d1-2a2c-4475-bb60-c8bba01b8df1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.159036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c66d1-2a2c-4475-bb60-c8bba01b8df1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.159312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c66d1-2a2c-4475-bb60-c8bba01b8df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.180993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-535c66d1-2a2c-4475-bb60-c8bba01b8df1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:10.182570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-535c66d1-2a2c-4475-bb60-c8bba01b8df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:10.215620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:10.229022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906c265-f594-4c25-a4e6-490d6125bae2 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:10.260962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 492/953] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:10.261802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c66d1-2a2c-4475-bb60-c8bba01b8df1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.273568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337f5d3-870b-4497-8138-d5ffc7b55a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:10.274067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337f5d3-870b-4497-8138-d5ffc7b55a35 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.274351 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337f5d3-870b-4497-8138-d5ffc7b55a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.274624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337f5d3-870b-4497-8138-d5ffc7b55a35 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.276031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337f5d3-870b-4497-8138-d5ffc7b55a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:10.276341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337f5d3-870b-4497-8138-d5ffc7b55a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:10.277746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337f5d3-870b-4497-8138-d5ffc7b55a35 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:10.294501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 462/954] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:10.294501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03f78d-dfd9-4dce-b0dc-d56b3e28109b None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.294501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03f78d-dfd9-4dce-b0dc-d56b3e28109b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.294501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03f78d-dfd9-4dce-b0dc-d56b3e28109b None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.294501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 463/955] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:10.294501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03f78d-dfd9-4dce-b0dc-d56b3e28109b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.375048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c66d1-2a2c-4475-bb60-c8bba01b8df1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:10.382609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:10.404102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03f78d-dfd9-4dce-b0dc-d56b3e28109b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:10.409886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:10.410548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.410833 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.411108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.411807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:10.413275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:10.414637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING keystone.server.flask.common [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 15:17:10.431470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.sql.core [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 15:17:10.431470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 15:17:10.431470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [parameters: {'id': 'bf180a997b474230bded238300bb584a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 15:17:10.431470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80422) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf180a997b474230bded238300bb584a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf180a997b474230bded238300bb584a. May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application conn.query(q) May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:17:10.438849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application result.read() May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 23 15:17:10.440157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return project_ref.to_dict() May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application next(self.gen) May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application yield resource May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application next(self.gen) May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.session.rollback() May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.force_reraise() May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise self.value May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application session.commit() May 23 15:17:10.441430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._prepare_impl() May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.session.flush() May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._flush(objects) May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application compat.raise_( May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application flush_context.execute() May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 15:17:10.442315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rec.execute(self) May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 15:17:10.443350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application conn.query(q) May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application result.read() May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:17:10.444136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [parameters: {'id': 'bf180a997b474230bded238300bb584a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.445032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:10.445876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.Conflict( May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf180a997b474230bded238300bb584a. May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:10.447366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 493/956] 10.222.0.30 () {60 vars in 1140 bytes} [Mon May 23 15:17:10 2022] POST /identity/v3/projects => generated 262 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:10.448070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7af8e1-ff21-4c68-bfa1-5d1ccccdf529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:10.448461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.448537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7af8e1-ff21-4c68-bfa1-5d1ccccdf529 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.448537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7af8e1-ff21-4c68-bfa1-5d1ccccdf529 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.448537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7af8e1-ff21-4c68-bfa1-5d1ccccdf529 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.448537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7af8e1-ff21-4c68-bfa1-5d1ccccdf529 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:10.448537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7af8e1-ff21-4c68-bfa1-5d1ccccdf529 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-9a7af8e1-ff21-4c68-bfa1-5d1ccccdf529 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:10.458359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:10.459523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:10.466232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 464/957] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:17:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:10.467121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7addd-fc3e-492f-8288-b5f4e8885552 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.467121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7addd-fc3e-492f-8288-b5f4e8885552 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.467121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7addd-fc3e-492f-8288-b5f4e8885552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.480041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-95b7addd-fc3e-492f-8288-b5f4e8885552 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:10.481334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-95b7addd-fc3e-492f-8288-b5f4e8885552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:10.491357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:10.546082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cb388-550a-4963-b4bf-78c232ee38e8 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:10.553451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 465/958] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:10.554343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7addd-fc3e-492f-8288-b5f4e8885552 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.580744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e66e5-9700-429a-9edd-db800304d219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:10.581594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e66e5-9700-429a-9edd-db800304d219 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.581879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e66e5-9700-429a-9edd-db800304d219 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.582158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e66e5-9700-429a-9edd-db800304d219 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.590606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e66e5-9700-429a-9edd-db800304d219 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:10.592241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e66e5-9700-429a-9edd-db800304d219 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-f26e66e5-9700-429a-9edd-db800304d219 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:17:10.607091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:10.608262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 494/959] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:17:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:10.609641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e66e5-9700-429a-9edd-db800304d219 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.635774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:10.671994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7addd-fc3e-492f-8288-b5f4e8885552 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:10.710363 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6b12c-c067-43ec-a74b-c3691d73cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:10.711138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6b12c-c067-43ec-a74b-c3691d73cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.711414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6b12c-c067-43ec-a74b-c3691d73cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.711682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6b12c-c067-43ec-a74b-c3691d73cbf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.718865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6b12c-c067-43ec-a74b-c3691d73cbf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:10.720317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6b12c-c067-43ec-a74b-c3691d73cbf7 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:10.721313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e66e5-9700-429a-9edd-db800304d219 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-7cc6b12c-c067-43ec-a74b-c3691d73cbf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:17:10.736657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:10.737953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 466/960] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:10.738961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6b12c-c067-43ec-a74b-c3691d73cbf7 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.747533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:10.760139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4070a-48e6-4600-9c07-2b269de9a40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:10.760806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4070a-48e6-4600-9c07-2b269de9a40d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.761117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4070a-48e6-4600-9c07-2b269de9a40d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.761389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4070a-48e6-4600-9c07-2b269de9a40d None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.762094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4070a-48e6-4600-9c07-2b269de9a40d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:10.762407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4070a-48e6-4600-9c07-2b269de9a40d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:10.763786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4070a-48e6-4600-9c07-2b269de9a40d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:10.771283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 495/961] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:10.772055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4070a-48e6-4600-9c07-2b269de9a40d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.847393 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6b12c-c067-43ec-a74b-c3691d73cbf7 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:10.856151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:10.871528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4070a-48e6-4600-9c07-2b269de9a40d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:10.881230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cba42c-ab26-42ed-bd8d-53a702bdef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:10.882023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cba42c-ab26-42ed-bd8d-53a702bdef06 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.882303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cba42c-ab26-42ed-bd8d-53a702bdef06 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.882599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cba42c-ab26-42ed-bd8d-53a702bdef06 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.883237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cba42c-ab26-42ed-bd8d-53a702bdef06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:10.884682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cba42c-ab26-42ed-bd8d-53a702bdef06 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-74cba42c-ab26-42ed-bd8d-53a702bdef06 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:10.897096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:10.898154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:10.899003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 467/962] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:10.899808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cba42c-ab26-42ed-bd8d-53a702bdef06 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.909525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f4c38-219a-4d4b-ad78-d7f1440bf1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:10.910244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4c38-219a-4d4b-ad78-d7f1440bf1ee None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:10.910534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4c38-219a-4d4b-ad78-d7f1440bf1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:10.910810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4c38-219a-4d4b-ad78-d7f1440bf1ee None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:10.911645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f4c38-219a-4d4b-ad78-d7f1440bf1ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:10.911954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f4c38-219a-4d4b-ad78-d7f1440bf1ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:10.913413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f4c38-219a-4d4b-ad78-d7f1440bf1ee None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:10.932249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 496/963] 10.222.0.30 () {56 vars in 1149 bytes} [Mon May 23 15:17:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:10.933238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f4c38-219a-4d4b-ad78-d7f1440bf1ee None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:10.969766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:11.006239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cba42c-ab26-42ed-bd8d-53a702bdef06 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:11.034237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f4c38-219a-4d4b-ad78-d7f1440bf1ee None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:11.039090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2299fd-2b99-423a-9316-b9d523f4d88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:11.039746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2299fd-2b99-423a-9316-b9d523f4d88c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:11.040021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2299fd-2b99-423a-9316-b9d523f4d88c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:11.040291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2299fd-2b99-423a-9316-b9d523f4d88c None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:11.041637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2299fd-2b99-423a-9316-b9d523f4d88c None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:11.042018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2299fd-2b99-423a-9316-b9d523f4d88c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:11.043435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2299fd-2b99-423a-9316-b9d523f4d88c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:11.059991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 468/964] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:17:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:11.060801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2299fd-2b99-423a-9316-b9d523f4d88c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:11.070957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130e2cd-7925-4ee1-8d53-17054a79f2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:11.071653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130e2cd-7925-4ee1-8d53-17054a79f2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:11.071984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130e2cd-7925-4ee1-8d53-17054a79f2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:11.072334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130e2cd-7925-4ee1-8d53-17054a79f2b4 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:11.073141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130e2cd-7925-4ee1-8d53-17054a79f2b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:11.073456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130e2cd-7925-4ee1-8d53-17054a79f2b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:11.074909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130e2cd-7925-4ee1-8d53-17054a79f2b4 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:11.086368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 497/965] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:11.086368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130e2cd-7925-4ee1-8d53-17054a79f2b4 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:11.097699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:11.199662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2299fd-2b99-423a-9316-b9d523f4d88c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:11.201607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130e2cd-7925-4ee1-8d53-17054a79f2b4 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:11.230443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:11.240501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0f522-9c91-4a86-bd95-87cad6b3eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:11.241300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0f522-9c91-4a86-bd95-87cad6b3eaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:11.241652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0f522-9c91-4a86-bd95-87cad6b3eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:11.242001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0f522-9c91-4a86-bd95-87cad6b3eaf6 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:11.243186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0f522-9c91-4a86-bd95-87cad6b3eaf6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:11.243186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0f522-9c91-4a86-bd95-87cad6b3eaf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:11.243400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f52016-9906-4285-879b-5f099898d348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:11.244207 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52016-9906-4285-879b-5f099898d348 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:11.244470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0f522-9c91-4a86-bd95-87cad6b3eaf6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:11.244703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52016-9906-4285-879b-5f099898d348 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:11.245064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52016-9906-4285-879b-5f099898d348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:11.253093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 469/966] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:17:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:11.253867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0f522-9c91-4a86-bd95-87cad6b3eaf6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:11.325929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f52016-9906-4285-879b-5f099898d348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:11.327396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f52016-9906-4285-879b-5f099898d348 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:11.337114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:11.373372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0f522-9c91-4a86-bd95-87cad6b3eaf6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:11.412106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e23a-b46d-4765-a45d-350e221b933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:11.412890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e23a-b46d-4765-a45d-350e221b933a None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:11.413253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e23a-b46d-4765-a45d-350e221b933a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:11.413597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e23a-b46d-4765-a45d-350e221b933a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:11.414536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2e23a-b46d-4765-a45d-350e221b933a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:11.416157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2e23a-b46d-4765-a45d-350e221b933a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:11.435922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 470/967] 10.222.0.30 () {60 vars in 1144 bytes} [Mon May 23 15:17:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:11.436720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 498/968] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:11 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:17:11.436991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e23a-b46d-4765-a45d-350e221b933a None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:11.446509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f52016-9906-4285-879b-5f099898d348 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:11.507873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:11.590958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f52016-9906-4285-879b-5f099898d348 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:11.591370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e23a-b46d-4765-a45d-350e221b933a None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:11.605690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:11.645001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3fbd3-12cb-4f23-be52-ff32d572e5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:11.645659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143d08a-ea3d-46ca-9e55-bde5d28a1070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:11.645804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3fbd3-12cb-4f23-be52-ff32d572e5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:11.646044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3fbd3-12cb-4f23-be52-ff32d572e5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:11.646324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3fbd3-12cb-4f23-be52-ff32d572e5f1 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:11.647047 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3fbd3-12cb-4f23-be52-ff32d572e5f1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:11.647272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143d08a-ea3d-46ca-9e55-bde5d28a1070 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:11.647416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3fbd3-12cb-4f23-be52-ff32d572e5f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:11.647576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143d08a-ea3d-46ca-9e55-bde5d28a1070 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:11.647836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143d08a-ea3d-46ca-9e55-bde5d28a1070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:11.648766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3fbd3-12cb-4f23-be52-ff32d572e5f1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:11.654986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 471/969] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:17:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:11.659659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143d08a-ea3d-46ca-9e55-bde5d28a1070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:11.661308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143d08a-ea3d-46ca-9e55-bde5d28a1070 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-b143d08a-ea3d-46ca-9e55-bde5d28a1070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:11.683991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:11.685107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:11.686092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:11.686092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:11.686092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:11.686092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:11.686092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 499/970] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:11.691321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3fbd3-12cb-4f23-be52-ff32d572e5f1 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:11.813065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3fbd3-12cb-4f23-be52-ff32d572e5f1 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:11.824492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:11.855127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be8bef-1f1a-4036-b78f-ecedb9ed901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:11.856186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be8bef-1f1a-4036-b78f-ecedb9ed901a None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:11.856675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be8bef-1f1a-4036-b78f-ecedb9ed901a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:11.857178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be8bef-1f1a-4036-b78f-ecedb9ed901a None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:11.858069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be8bef-1f1a-4036-b78f-ecedb9ed901a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:11.858382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be8bef-1f1a-4036-b78f-ecedb9ed901a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:11.859769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be8bef-1f1a-4036-b78f-ecedb9ed901a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:11.873701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 472/971] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:11.879078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143d08a-ea3d-46ca-9e55-bde5d28a1070 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:11.996844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143d08a-ea3d-46ca-9e55-bde5d28a1070 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:12.033437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6b6f4-d1f3-4a9a-b5d1-b4ec49ce821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:12.034238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6b6f4-d1f3-4a9a-b5d1-b4ec49ce821b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:12.034520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6b6f4-d1f3-4a9a-b5d1-b4ec49ce821b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:12.034794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6b6f4-d1f3-4a9a-b5d1-b4ec49ce821b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:12.042427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6b6f4-d1f3-4a9a-b5d1-b4ec49ce821b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:12.043872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6b6f4-d1f3-4a9a-b5d1-b4ec49ce821b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-a5c6b6f4-d1f3-4a9a-b5d1-b4ec49ce821b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:17:12.060215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:12.061158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:12.062058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:17:12.062058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:17:12.062058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:17:12.062058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:17:12.062058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:17:12.062058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:17:12.062058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:12.062058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:12.062058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 500/972] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:12.068803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be8bef-1f1a-4036-b78f-ecedb9ed901a None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:12.173502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be8bef-1f1a-4036-b78f-ecedb9ed901a None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:12.209934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90005b6c-d508-46dc-84dd-7dc8f617a45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:12.210954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005b6c-d508-46dc-84dd-7dc8f617a45c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:12.211471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005b6c-d508-46dc-84dd-7dc8f617a45c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:12.211947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005b6c-d508-46dc-84dd-7dc8f617a45c None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:12.213063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90005b6c-d508-46dc-84dd-7dc8f617a45c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:12.213606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90005b6c-d508-46dc-84dd-7dc8f617a45c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:12.215247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90005b6c-d508-46dc-84dd-7dc8f617a45c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:12.227807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 473/973] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:17:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:12.234588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6b6f4-d1f3-4a9a-b5d1-b4ec49ce821b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:12.259201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:12.347762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6b6f4-d1f3-4a9a-b5d1-b4ec49ce821b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:12.382873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de7a03-c71d-4c64-8c15-742158958864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:12.383542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de7a03-c71d-4c64-8c15-742158958864 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:12.383830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de7a03-c71d-4c64-8c15-742158958864 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:12.384112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de7a03-c71d-4c64-8c15-742158958864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:12.385044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de7a03-c71d-4c64-8c15-742158958864 None admin] RBAC: Adding query filter params (role.id=e3f9cda0e462469f92d47a2624887c97, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=0bb6b4503a4e43f480af8430e7f4fc49) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:12.385365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de7a03-c71d-4c64-8c15-742158958864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:12.386804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de7a03-c71d-4c64-8c15-742158958864 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:12.393409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 501/974] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:17:12 2022] GET /identity/v3/role_assignments?user.id=0bb6b4503a4e43f480af8430e7f4fc49&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=e3f9cda0e462469f92d47a2624887c97 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:13.379855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc753-4c45-4d19-99cf-4794dfb484bb None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:13.380400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc753-4c45-4d19-99cf-4794dfb484bb None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:13.380400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc753-4c45-4d19-99cf-4794dfb484bb None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:13.382179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 474/975] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:13.388438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c7825-8540-482c-8a77-f40fa4fd8018 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:13.388896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c7825-8540-482c-8a77-f40fa4fd8018 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:13.389095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c7825-8540-482c-8a77-f40fa4fd8018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:13.406204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1b5c7825-8540-482c-8a77-f40fa4fd8018 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:13.407710 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1b5c7825-8540-482c-8a77-f40fa4fd8018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:13.480804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 502/976] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:13.501577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcc753-4c45-4d19-99cf-4794dfb484bb None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:13.530240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:13.612905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcc753-4c45-4d19-99cf-4794dfb484bb None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:13.644837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:13.645456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:13.645735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:13.646005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:13.646712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:13.648151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:13.649479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WARNING keystone.server.flask.common [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 15:17:13.662542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.sql.core [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 15:17:13.662542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 15:17:13.662542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [parameters: {'user_id': '422bbdf80a5f4bb9ac3d73a9b8da8bc2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 15:17:13.662542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80421) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application conn.query(q) May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application result.read() May 23 15:17:13.679413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:17:13.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application next(self.gen) May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application yield resource May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application next(self.gen) May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self.session.rollback() May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self.force_reraise() May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise self.value May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application session.commit() May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 15:17:13.681438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._prepare_impl() May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self.session.flush() May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._flush(objects) May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application compat.raise_( May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application flush_context.execute() May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rec.execute(self) May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 15:17:13.682631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application result = connection._execute_20( May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:17:13.683556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application conn.query(q) May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application result.read() May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:17:13.684402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [parameters: {'user_id': '422bbdf80a5f4bb9ac3d73a9b8da8bc2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:13.686902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 23 15:17:13.688913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 23 15:17:13.693167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 23 15:17:13.693167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 23 15:17:13.693167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 15:17:13.693167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 15:17:13.693167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 15:17:13.693167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:13.693167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 475/977] 10.222.0.30 () {60 vars in 1131 bytes} [Mon May 23 15:17:13 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:13.693616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c7825-8540-482c-8a77-f40fa4fd8018 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:13.717286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:13.803859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c7825-8540-482c-8a77-f40fa4fd8018 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:13.844079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2012847-5b50-45a6-8c89-242f150efe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:13.844907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2012847-5b50-45a6-8c89-242f150efe3d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:13.845195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2012847-5b50-45a6-8c89-242f150efe3d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:13.845471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2012847-5b50-45a6-8c89-242f150efe3d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:13.853698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2012847-5b50-45a6-8c89-242f150efe3d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:13.855144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2012847-5b50-45a6-8c89-242f150efe3d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-b2012847-5b50-45a6-8c89-242f150efe3d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:17:13.874227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:13.875452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:13.876650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 503/978] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:17:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:13.883623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:13.906189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789481c-0354-4950-9505-07cd48cd0dd5 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:13.906379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789481c-0354-4950-9505-07cd48cd0dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:13.906666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789481c-0354-4950-9505-07cd48cd0dd5 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:13.908322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 504/979] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:17:13.913684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614b22-eeed-48e1-98fe-548f10de099e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:13.913978 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614b22-eeed-48e1-98fe-548f10de099e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:13.914262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614b22-eeed-48e1-98fe-548f10de099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:13.932642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b3614b22-eeed-48e1-98fe-548f10de099e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:13.933977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b3614b22-eeed-48e1-98fe-548f10de099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:13.982580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:13.994610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023c787-2e57-47b9-926b-1b7ba6cf1d36 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:14.015822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 505/980] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:14.017764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78970c5c-ad39-4adf-a462-bbf551d9259a None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.018229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78970c5c-ad39-4adf-a462-bbf551d9259a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.018686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78970c5c-ad39-4adf-a462-bbf551d9259a None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.020454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 506/981] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:14.025851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18248098-5283-4bf4-973f-169261eebc58 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.026366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18248098-5283-4bf4-973f-169261eebc58 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.026853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18248098-5283-4bf4-973f-169261eebc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.031636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403855d-f234-457f-a2f2-a2063af84f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:14.032304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403855d-f234-457f-a2f2-a2063af84f27 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.032587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403855d-f234-457f-a2f2-a2063af84f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.032861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403855d-f234-457f-a2f2-a2063af84f27 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.033681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403855d-f234-457f-a2f2-a2063af84f27 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:14.034166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:14.034320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403855d-f234-457f-a2f2-a2063af84f27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:14.036298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403855d-f234-457f-a2f2-a2063af84f27 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:14.046895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-18248098-5283-4bf4-973f-169261eebc58 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:14.048286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-18248098-5283-4bf4-973f-169261eebc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:14.076979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 476/982] 10.222.0.30 () {56 vars in 1101 bytes} [Mon May 23 15:17:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:14.077830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403855d-f234-457f-a2f2-a2063af84f27 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:14.157178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 507/983] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:14.177080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18248098-5283-4bf4-973f-169261eebc58 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:14.207804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403855d-f234-457f-a2f2-a2063af84f27 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:14.258991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa81d1a-5adb-42cc-940f-39bd38a14278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:14.261141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa81d1a-5adb-42cc-940f-39bd38a14278 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.261491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa81d1a-5adb-42cc-940f-39bd38a14278 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.262029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa81d1a-5adb-42cc-940f-39bd38a14278 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.262732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa81d1a-5adb-42cc-940f-39bd38a14278 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:14.264929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa81d1a-5adb-42cc-940f-39bd38a14278 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-3aa81d1a-5adb-42cc-940f-39bd38a14278 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:14.283534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:14.284581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:14.285622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 477/984] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:17:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:14.291850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa81d1a-5adb-42cc-940f-39bd38a14278 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:14.314345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:14.315496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18248098-5283-4bf4-973f-169261eebc58 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:14.367374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c75a15-67ae-4d41-91ce-fe63cff7da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:14.368765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c75a15-67ae-4d41-91ce-fe63cff7da31 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.369383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c75a15-67ae-4d41-91ce-fe63cff7da31 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.369864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c75a15-67ae-4d41-91ce-fe63cff7da31 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.370814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c75a15-67ae-4d41-91ce-fe63cff7da31 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:14.372907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c75a15-67ae-4d41-91ce-fe63cff7da31 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:14.375789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:14.392838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 508/985] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:17:14 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:17:14.426285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa81d1a-5adb-42cc-940f-39bd38a14278 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:14.476257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c39a6-6d34-4e62-a310-be18e9c4d32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:14.476963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c39a6-6d34-4e62-a310-be18e9c4d32e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.477243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c39a6-6d34-4e62-a310-be18e9c4d32e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.477511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c39a6-6d34-4e62-a310-be18e9c4d32e None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.478244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c39a6-6d34-4e62-a310-be18e9c4d32e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:14.478551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c39a6-6d34-4e62-a310-be18e9c4d32e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:14.479992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c39a6-6d34-4e62-a310-be18e9c4d32e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:14.485909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 478/986] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:14.491400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c75a15-67ae-4d41-91ce-fe63cff7da31 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:14.611965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c75a15-67ae-4d41-91ce-fe63cff7da31 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:14.649021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe4a2-f5b4-439c-9b0c-e956a40b21fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:14.649690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe4a2-f5b4-439c-9b0c-e956a40b21fd None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.649970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe4a2-f5b4-439c-9b0c-e956a40b21fd None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.650371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe4a2-f5b4-439c-9b0c-e956a40b21fd None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.651284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe4a2-f5b4-439c-9b0c-e956a40b21fd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:14.651589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe4a2-f5b4-439c-9b0c-e956a40b21fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:14.652975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe4a2-f5b4-439c-9b0c-e956a40b21fd None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:14.658031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 509/987] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:14.663976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c39a6-6d34-4e62-a310-be18e9c4d32e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:14.734741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:14.738693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b81b8-eb72-420c-9363-486d9bf0ceac None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.739018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b81b8-eb72-420c-9363-486d9bf0ceac None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.739333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b81b8-eb72-420c-9363-486d9bf0ceac None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.741018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 510/988] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:17:14.746926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afad5f-4f0a-4008-8f1e-de5f3bd6eb84 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.747263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afad5f-4f0a-4008-8f1e-de5f3bd6eb84 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.747582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afad5f-4f0a-4008-8f1e-de5f3bd6eb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.766171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-81afad5f-4f0a-4008-8f1e-de5f3bd6eb84 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:14.767470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-81afad5f-4f0a-4008-8f1e-de5f3bd6eb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:14.768281 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c39a6-6d34-4e62-a310-be18e9c4d32e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:14.809452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0d59d-c03d-411c-adc3-fcb60e7eecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:14.810114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0d59d-c03d-411c-adc3-fcb60e7eecc2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.810395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0d59d-c03d-411c-adc3-fcb60e7eecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.810665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0d59d-c03d-411c-adc3-fcb60e7eecc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.811579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0d59d-c03d-411c-adc3-fcb60e7eecc2 None admin] RBAC: Adding query filter params (interface=admin, service_id=f75712e66aa14c8eb8b997ad6985dc2f, region_id=RegionOne) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:14.811887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0d59d-c03d-411c-adc3-fcb60e7eecc2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:14.813455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0d59d-c03d-411c-adc3-fcb60e7eecc2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:14.819069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 479/989] 10.222.0.30 () {56 vars in 1249 bytes} [Mon May 23 15:17:14 2022] GET /identity/v3/endpoints?service_id=f75712e66aa14c8eb8b997ad6985dc2f&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:14.822040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:14.825421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0d59d-c03d-411c-adc3-fcb60e7eecc2 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:14.839312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 511/990] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:14.861252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afad5f-4f0a-4008-8f1e-de5f3bd6eb84 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:14.932703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0d59d-c03d-411c-adc3-fcb60e7eecc2 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:14.969714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c8558-9372-4a37-bd4c-c9f44c87a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:14.970638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c8558-9372-4a37-bd4c-c9f44c87a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:14.970922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c8558-9372-4a37-bd4c-c9f44c87a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:14.971207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c8558-9372-4a37-bd4c-c9f44c87a5a1 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:14.971873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c8558-9372-4a37-bd4c-c9f44c87a5a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:14.973052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afad5f-4f0a-4008-8f1e-de5f3bd6eb84 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:14.973305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c8558-9372-4a37-bd4c-c9f44c87a5a1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:14.975108 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:14.980949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 480/991] 10.222.0.30 () {56 vars in 1087 bytes} [Mon May 23 15:17:14 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:17:15.018539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855d3e-f844-4260-ad44-a8549b603532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:15.019360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855d3e-f844-4260-ad44-a8549b603532 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:15.019643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855d3e-f844-4260-ad44-a8549b603532 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:15.019919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855d3e-f844-4260-ad44-a8549b603532 None admin] PATH_INFO: `/v3/services/image` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:15.020589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855d3e-f844-4260-ad44-a8549b603532 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:15.022023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855d3e-f844-4260-ad44-a8549b603532 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-c0855d3e-f844-4260-ad44-a8549b603532 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:15.035159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:15.036240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:15.037136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 512/992] 10.222.0.30 () {56 vars in 1105 bytes} [Mon May 23 15:17:14 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:15.042792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c8558-9372-4a37-bd4c-c9f44c87a5a1 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:15.158570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c8558-9372-4a37-bd4c-c9f44c87a5a1 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:15.196138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b8928-8187-45aa-9a21-2eb35d0cf1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:15.196809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b8928-8187-45aa-9a21-2eb35d0cf1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:15.197717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b8928-8187-45aa-9a21-2eb35d0cf1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:15.198081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b8928-8187-45aa-9a21-2eb35d0cf1c9 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:15.198818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b8928-8187-45aa-9a21-2eb35d0cf1c9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:15.199122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b8928-8187-45aa-9a21-2eb35d0cf1c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:15.200489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b8928-8187-45aa-9a21-2eb35d0cf1c9 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:15.205539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 481/993] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:15.211423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855d3e-f844-4260-ad44-a8549b603532 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:15.265714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:15.316081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855d3e-f844-4260-ad44-a8549b603532 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:15.351970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4cb3e-ffca-4bca-80e9-4346088a8c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:15.352639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4cb3e-ffca-4bca-80e9-4346088a8c63 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:15.352951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4cb3e-ffca-4bca-80e9-4346088a8c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:15.353274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4cb3e-ffca-4bca-80e9-4346088a8c63 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:15.353984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4cb3e-ffca-4bca-80e9-4346088a8c63 None admin] RBAC: Adding query filter params (type=image) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:15.354293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4cb3e-ffca-4bca-80e9-4346088a8c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:15.355655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4cb3e-ffca-4bca-80e9-4346088a8c63 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:15.369468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 513/994] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:16.679703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610f04a-a257-4c95-ab3c-19a80e754b35 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:16.680217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610f04a-a257-4c95-ab3c-19a80e754b35 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:16.680294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610f04a-a257-4c95-ab3c-19a80e754b35 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:16.681996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 482/995] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:16.686970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ec0e-8a5d-4a56-acd5-50adc6f9ea7e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:16.687373 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ec0e-8a5d-4a56-acd5-50adc6f9ea7e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:16.687553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ec0e-8a5d-4a56-acd5-50adc6f9ea7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:16.704926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ea33ec0e-8a5d-4a56-acd5-50adc6f9ea7e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:16.706102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ea33ec0e-8a5d-4a56-acd5-50adc6f9ea7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:16.776164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 514/996] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:16.798082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610f04a-a257-4c95-ab3c-19a80e754b35 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:16.861982 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:16.904192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610f04a-a257-4c95-ab3c-19a80e754b35 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:16.937902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff531b-24b7-404d-ba4d-ab8dc4ff7e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:16.938693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff531b-24b7-404d-ba4d-ab8dc4ff7e7c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:16.938972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff531b-24b7-404d-ba4d-ab8dc4ff7e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:16.939244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff531b-24b7-404d-ba4d-ab8dc4ff7e7c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:16.939890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff531b-24b7-404d-ba4d-ab8dc4ff7e7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:16.941296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff531b-24b7-404d-ba4d-ab8dc4ff7e7c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-72ff531b-24b7-404d-ba4d-ab8dc4ff7e7c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:16.953486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:16.954439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:17:16.955257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:16.957225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 483/997] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:16.960527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33ec0e-8a5d-4a56-acd5-50adc6f9ea7e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:17.006855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:17.066306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33ec0e-8a5d-4a56-acd5-50adc6f9ea7e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:17.101058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981a7fb-efa3-4633-92f4-0f3aa03d1836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:17.101733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981a7fb-efa3-4633-92f4-0f3aa03d1836 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.102015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981a7fb-efa3-4633-92f4-0f3aa03d1836 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.102291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981a7fb-efa3-4633-92f4-0f3aa03d1836 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.102997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981a7fb-efa3-4633-92f4-0f3aa03d1836 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:17.103304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981a7fb-efa3-4633-92f4-0f3aa03d1836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:17.104704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981a7fb-efa3-4633-92f4-0f3aa03d1836 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:17.110001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 515/998] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:17:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:17.115419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff531b-24b7-404d-ba4d-ab8dc4ff7e7c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:17.217667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff531b-24b7-404d-ba4d-ab8dc4ff7e7c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:17.218566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a41a8-cfa2-465c-9f92-317287216ee8 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.218854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a41a8-cfa2-465c-9f92-317287216ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.219138 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a41a8-cfa2-465c-9f92-317287216ee8 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.220656 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 516/999] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:17:17.225422 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadad02-c7c8-4436-884d-001f72f7d2e6 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.225713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadad02-c7c8-4436-884d-001f72f7d2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.225992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadad02-c7c8-4436-884d-001f72f7d2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.242661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0eadad02-c7c8-4436-884d-001f72f7d2e6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:17.243835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0eadad02-c7c8-4436-884d-001f72f7d2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:17.247693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:17.257207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aec5d-823a-4582-804d-da2d7a50ca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:17.257880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aec5d-823a-4582-804d-da2d7a50ca92 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.258158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aec5d-823a-4582-804d-da2d7a50ca92 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.258428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aec5d-823a-4582-804d-da2d7a50ca92 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.259139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8aec5d-823a-4582-804d-da2d7a50ca92 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:17.259445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8aec5d-823a-4582-804d-da2d7a50ca92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:17.260957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8aec5d-823a-4582-804d-da2d7a50ca92 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:17.266149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 484/1000] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:17:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:17.271554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aec5d-823a-4582-804d-da2d7a50ca92 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:17.318989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 517/1001] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:17.341431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadad02-c7c8-4436-884d-001f72f7d2e6 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:17.381255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aec5d-823a-4582-804d-da2d7a50ca92 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:17.391507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:17.422401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0f005-e520-41ba-98ad-39d7aadd2ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:17.423232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0f005-e520-41ba-98ad-39d7aadd2ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.424299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0f005-e520-41ba-98ad-39d7aadd2ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.424585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0f005-e520-41ba-98ad-39d7aadd2ed1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.425410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0f005-e520-41ba-98ad-39d7aadd2ed1 None admin] RBAC: Adding query filter params (interface=public, service_id=89c64704d6604922b3b3f92d192f7bb0, region_id=RegionOne) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:17.425727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0f005-e520-41ba-98ad-39d7aadd2ed1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:17.427105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0f005-e520-41ba-98ad-39d7aadd2ed1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:17.433909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 485/1002] 10.222.0.30 () {56 vars in 1251 bytes} [Mon May 23 15:17:17 2022] GET /identity/v3/endpoints?service_id=89c64704d6604922b3b3f92d192f7bb0&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:17.439035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0f005-e520-41ba-98ad-39d7aadd2ed1 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:17.454387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadad02-c7c8-4436-884d-001f72f7d2e6 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:17.490754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f72bf-8be1-4980-a6aa-3544332473f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:17.491567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f72bf-8be1-4980-a6aa-3544332473f3 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.491849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f72bf-8be1-4980-a6aa-3544332473f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.492124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f72bf-8be1-4980-a6aa-3544332473f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.492782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f72bf-8be1-4980-a6aa-3544332473f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:17.494773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f72bf-8be1-4980-a6aa-3544332473f3 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:17.497731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-582f72bf-8be1-4980-a6aa-3544332473f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:17.507214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:17.508233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:17.509432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 518/1003] 10.222.0.30 () {56 vars in 1126 bytes} [Mon May 23 15:17:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:17.515034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f72bf-8be1-4980-a6aa-3544332473f3 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:17.549787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0f005-e520-41ba-98ad-39d7aadd2ed1 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:17.585921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9d92e-6c19-45c6-9e52-311b2f934f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:17.587078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d92e-6c19-45c6-9e52-311b2f934f32 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.587078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d92e-6c19-45c6-9e52-311b2f934f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.587270 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d92e-6c19-45c6-9e52-311b2f934f32 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.587863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9d92e-6c19-45c6-9e52-311b2f934f32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:17.589373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9d92e-6c19-45c6-9e52-311b2f934f32 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:17.595708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 486/1004] 10.222.0.30 () {56 vars in 1087 bytes} [Mon May 23 15:17:17 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:17:17.596975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b4952-1fe8-4e2d-980f-1ad8a5a46fce None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.597263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b4952-1fe8-4e2d-980f-1ad8a5a46fce None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.597543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b4952-1fe8-4e2d-980f-1ad8a5a46fce None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.598799 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 487/1005] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:17.604661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058cfc-a056-4f60-a87c-9a8b9c9b8fa7 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.605007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058cfc-a056-4f60-a87c-9a8b9c9b8fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.605247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058cfc-a056-4f60-a87c-9a8b9c9b8fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.624242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-5d058cfc-a056-4f60-a87c-9a8b9c9b8fa7 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:17.626121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5d058cfc-a056-4f60-a87c-9a8b9c9b8fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:17.629609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f72bf-8be1-4980-a6aa-3544332473f3 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:17.633212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:17.672544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18d9e5-6ee5-41c2-a46f-e5f8ce0a8a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:17.673400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18d9e5-6ee5-41c2-a46f-e5f8ce0a8a57 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.673687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18d9e5-6ee5-41c2-a46f-e5f8ce0a8a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.673971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18d9e5-6ee5-41c2-a46f-e5f8ce0a8a57 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.674899 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18d9e5-6ee5-41c2-a46f-e5f8ce0a8a57 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:17.675244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18d9e5-6ee5-41c2-a46f-e5f8ce0a8a57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:17.676725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18d9e5-6ee5-41c2-a46f-e5f8ce0a8a57 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:17.684107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 519/1006] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:17.691856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18d9e5-6ee5-41c2-a46f-e5f8ce0a8a57 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:17.712806 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 488/1007] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:17.735794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d058cfc-a056-4f60-a87c-9a8b9c9b8fa7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:17.795238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:17.823435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18d9e5-6ee5-41c2-a46f-e5f8ce0a8a57 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:17.852888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d058cfc-a056-4f60-a87c-9a8b9c9b8fa7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:17.859528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b01e9d-d47d-452b-b5c5-f6e06209b129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:17.860209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b01e9d-d47d-452b-b5c5-f6e06209b129 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.860903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b01e9d-d47d-452b-b5c5-f6e06209b129 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.860903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b01e9d-d47d-452b-b5c5-f6e06209b129 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.861606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b01e9d-d47d-452b-b5c5-f6e06209b129 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:17.861914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b01e9d-d47d-452b-b5c5-f6e06209b129 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:17.863391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b01e9d-d47d-452b-b5c5-f6e06209b129 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:17.868578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 520/1008] 10.222.0.30 () {56 vars in 1122 bytes} [Mon May 23 15:17:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:17.874582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b01e9d-d47d-452b-b5c5-f6e06209b129 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:17.891171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7058c-108a-4be6-b842-0c8cd9730943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:17.892144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7058c-108a-4be6-b842-0c8cd9730943 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:17.892422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7058c-108a-4be6-b842-0c8cd9730943 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:17.892838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7058c-108a-4be6-b842-0c8cd9730943 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:17.893586 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7058c-108a-4be6-b842-0c8cd9730943 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:17.894966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7058c-108a-4be6-b842-0c8cd9730943 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:17.908658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:17.909896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 489/1009] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:17:17 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:17:17.992718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b01e9d-d47d-452b-b5c5-f6e06209b129 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:18.037862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1765fb8-10d6-4671-a986-05cd2b44f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:18.038555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1765fb8-10d6-4671-a986-05cd2b44f32c None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:18.038833 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1765fb8-10d6-4671-a986-05cd2b44f32c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:18.039105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1765fb8-10d6-4671-a986-05cd2b44f32c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:18.039878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1765fb8-10d6-4671-a986-05cd2b44f32c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:18.041533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1765fb8-10d6-4671-a986-05cd2b44f32c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:18.064904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 521/1010] 10.222.0.30 () {60 vars in 1144 bytes} [Mon May 23 15:17:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:17:19.852211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d9f4e-ba38-44e6-96e5-39d8ebcef466 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:19.852725 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d9f4e-ba38-44e6-96e5-39d8ebcef466 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:19.852801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d9f4e-ba38-44e6-96e5-39d8ebcef466 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:19.854595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 490/1011] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:17:19.859771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016711d5-97f0-41a4-9c75-b44c98b9f9b0 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:19.860176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016711d5-97f0-41a4-9c75-b44c98b9f9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:19.860252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016711d5-97f0-41a4-9c75-b44c98b9f9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:19.877818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-016711d5-97f0-41a4-9c75-b44c98b9f9b0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:19.879028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-016711d5-97f0-41a4-9c75-b44c98b9f9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:19.954374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 522/1012] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:19 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:19.976560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d9f4e-ba38-44e6-96e5-39d8ebcef466 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.084900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d9f4e-ba38-44e6-96e5-39d8ebcef466 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:20.098289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:20.126031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d0b63-2ecd-4ca1-9148-b88b820c00aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:20.126749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d0b63-2ecd-4ca1-9148-b88b820c00aa None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.127030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d0b63-2ecd-4ca1-9148-b88b820c00aa None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.127303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d0b63-2ecd-4ca1-9148-b88b820c00aa None admin] PATH_INFO: `/v3/services/compute` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.127961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d0b63-2ecd-4ca1-9148-b88b820c00aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:20.129364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d0b63-2ecd-4ca1-9148-b88b820c00aa None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b45d0b63-2ecd-4ca1-9148-b88b820c00aa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:20.141736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:20.142753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:17:20.143699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:20.151090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 491/1013] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:20.151223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016711d5-97f0-41a4-9c75-b44c98b9f9b0 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.164583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:20.188974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c465f-5c5a-4818-9ac6-8296a27c9133 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.189279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c465f-5c5a-4818-9ac6-8296a27c9133 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.189529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c465f-5c5a-4818-9ac6-8296a27c9133 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.191046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 492/1014] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:17:20.195829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0bfd8-14cf-4c3b-a498-b60706300df8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.196113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0bfd8-14cf-4c3b-a498-b60706300df8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.196393 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0bfd8-14cf-4c3b-a498-b60706300df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.213296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-03d0bfd8-14cf-4c3b-a498-b60706300df8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:20.214468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-03d0bfd8-14cf-4c3b-a498-b60706300df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:20.254609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016711d5-97f0-41a4-9c75-b44c98b9f9b0 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:20.289502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e2dff-9d9a-44f3-a31f-7386ec43ead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:20.290178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e2dff-9d9a-44f3-a31f-7386ec43ead8 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.290460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e2dff-9d9a-44f3-a31f-7386ec43ead8 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.290732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e2dff-9d9a-44f3-a31f-7386ec43ead8 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.290858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 493/1015] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:20 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:20.291972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e2dff-9d9a-44f3-a31f-7386ec43ead8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:20.292275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e2dff-9d9a-44f3-a31f-7386ec43ead8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:20.292861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049742c-71d8-4f38-8ab7-2dded5e583d5 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.293169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049742c-71d8-4f38-8ab7-2dded5e583d5 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.293439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049742c-71d8-4f38-8ab7-2dded5e583d5 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.293673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e2dff-9d9a-44f3-a31f-7386ec43ead8 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:20.295217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 494/1016] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:20.304944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf140ab2-7848-49b7-80da-efe559349bbb None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.305086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 523/1017] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:17:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:20.305178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf140ab2-7848-49b7-80da-efe559349bbb None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.305178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf140ab2-7848-49b7-80da-efe559349bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.305767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e2dff-9d9a-44f3-a31f-7386ec43ead8 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.318581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-cf140ab2-7848-49b7-80da-efe559349bbb None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:20.319762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-cf140ab2-7848-49b7-80da-efe559349bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:20.388746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 495/1018] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:20 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:20.389555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf140ab2-7848-49b7-80da-efe559349bbb None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.389782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:20.412738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e2dff-9d9a-44f3-a31f-7386ec43ead8 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:20.423760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:20.447962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d7760-fc22-4246-a243-4a7c7637f467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:20.448624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d7760-fc22-4246-a243-4a7c7637f467 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.448938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d7760-fc22-4246-a243-4a7c7637f467 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.449215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d7760-fc22-4246-a243-4a7c7637f467 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.449912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d7760-fc22-4246-a243-4a7c7637f467 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:20.450215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d7760-fc22-4246-a243-4a7c7637f467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:20.451574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d7760-fc22-4246-a243-4a7c7637f467 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:20.456317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 524/1019] 10.222.0.30 () {56 vars in 1112 bytes} [Mon May 23 15:17:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:20.457125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d7760-fc22-4246-a243-4a7c7637f467 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.489003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf140ab2-7848-49b7-80da-efe559349bbb None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:20.522305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ec95-1845-49df-8c48-d26f02ff4989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:20.523093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ec95-1845-49df-8c48-d26f02ff4989 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.523373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ec95-1845-49df-8c48-d26f02ff4989 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.523648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ec95-1845-49df-8c48-d26f02ff4989 None admin] PATH_INFO: `/v3/services/image` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.524298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790ec95-1845-49df-8c48-d26f02ff4989 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:20.525818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790ec95-1845-49df-8c48-d26f02ff4989 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b790ec95-1845-49df-8c48-d26f02ff4989 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:20.537380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:20.538468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:20.539293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 496/1020] 10.222.0.30 () {56 vars in 1105 bytes} [Mon May 23 15:17:20 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:20.540066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ec95-1845-49df-8c48-d26f02ff4989 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.557855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d7760-fc22-4246-a243-4a7c7637f467 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:20.591280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d662a97-16ad-4d26-ac48-dc44977dde81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:20.592068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662a97-16ad-4d26-ac48-dc44977dde81 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.592348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662a97-16ad-4d26-ac48-dc44977dde81 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.592620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662a97-16ad-4d26-ac48-dc44977dde81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.593821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d662a97-16ad-4d26-ac48-dc44977dde81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:20.595250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d662a97-16ad-4d26-ac48-dc44977dde81 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:20.596857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 525/1021] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:20.597665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d662a97-16ad-4d26-ac48-dc44977dde81 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.611391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:20.640163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:20.644233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ec95-1845-49df-8c48-d26f02ff4989 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:20.678754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8247ff1-88b8-4788-911d-214d2f994603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:20.679399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247ff1-88b8-4788-911d-214d2f994603 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.679716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247ff1-88b8-4788-911d-214d2f994603 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.679978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247ff1-88b8-4788-911d-214d2f994603 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.680708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8247ff1-88b8-4788-911d-214d2f994603 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:20.682153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8247ff1-88b8-4788-911d-214d2f994603 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:20.700150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d662a97-16ad-4d26-ac48-dc44977dde81 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:20.700630 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 497/1022] 10.222.0.30 () {60 vars in 1144 bytes} [Mon May 23 15:17:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:20.701467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8247ff1-88b8-4788-911d-214d2f994603 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.763737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a1d5-8440-4b49-bba7-6c4905a86957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:20.764426 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a1d5-8440-4b49-bba7-6c4905a86957 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.764709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a1d5-8440-4b49-bba7-6c4905a86957 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.765020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a1d5-8440-4b49-bba7-6c4905a86957 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.765750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30a1d5-8440-4b49-bba7-6c4905a86957 None admin] RBAC: Adding query filter params (name=image) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:20.766057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30a1d5-8440-4b49-bba7-6c4905a86957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:20.767468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30a1d5-8440-4b49-bba7-6c4905a86957 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:20.773375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 526/1023] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:20.779876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a1d5-8440-4b49-bba7-6c4905a86957 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.828796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8247ff1-88b8-4788-911d-214d2f994603 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:20.857876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:20.869726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c899e-bf16-480f-83fa-f98f356e614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:20.870689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c899e-bf16-480f-83fa-f98f356e614d None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.870972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c899e-bf16-480f-83fa-f98f356e614d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.871240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c899e-bf16-480f-83fa-f98f356e614d None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.872137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c899e-bf16-480f-83fa-f98f356e614d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:20.874394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c899e-bf16-480f-83fa-f98f356e614d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:20.892127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a1d5-8440-4b49-bba7-6c4905a86957 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:20.911048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 498/1024] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:17:20 2022] POST /identity/v3/projects => generated 311 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:20.930894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509ff19-9739-48b2-8ae4-e16f973b28a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:20.931593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ff19-9739-48b2-8ae4-e16f973b28a1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:20.931975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ff19-9739-48b2-8ae4-e16f973b28a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:20.932247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ff19-9739-48b2-8ae4-e16f973b28a1 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:20.933004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509ff19-9739-48b2-8ae4-e16f973b28a1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:20.933308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509ff19-9739-48b2-8ae4-e16f973b28a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:20.934707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509ff19-9739-48b2-8ae4-e16f973b28a1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:20.942055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 527/1025] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:20.949254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c899e-bf16-480f-83fa-f98f356e614d None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:20.959493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:21.083472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c899e-bf16-480f-83fa-f98f356e614d None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:21.121911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebeaad-1f19-4a9b-bcc5-553743449c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:21.122576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebeaad-1f19-4a9b-bcc5-553743449c0a None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:21.122859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebeaad-1f19-4a9b-bcc5-553743449c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:21.123129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebeaad-1f19-4a9b-bcc5-553743449c0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:21.123918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebeaad-1f19-4a9b-bcc5-553743449c0a None admin] RBAC: Adding query filter params (interface=public, service_id=d2c386f9d1594e14b75bf7aceb82d5b1, region_id=RegionOne) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:21.124230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebeaad-1f19-4a9b-bcc5-553743449c0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:21.125901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebeaad-1f19-4a9b-bcc5-553743449c0a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:21.131401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 499/1026] 10.222.0.30 () {56 vars in 1251 bytes} [Mon May 23 15:17:20 2022] GET /identity/v3/endpoints?service_id=d2c386f9d1594e14b75bf7aceb82d5b1&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:21.137696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509ff19-9739-48b2-8ae4-e16f973b28a1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:21.244527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509ff19-9739-48b2-8ae4-e16f973b28a1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:21.279929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd641f-fa19-4387-884d-30e34d48d0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:21.280636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd641f-fa19-4387-884d-30e34d48d0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:21.281024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd641f-fa19-4387-884d-30e34d48d0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:21.281226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd641f-fa19-4387-884d-30e34d48d0f3 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:21.281906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd641f-fa19-4387-884d-30e34d48d0f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:21.283267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd641f-fa19-4387-884d-30e34d48d0f3 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:21.284392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:21.297273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 528/1027] 10.222.0.30 () {56 vars in 1087 bytes} [Mon May 23 15:17:21 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:17:23.104956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762eea2-cc74-4ed1-aea4-dbc8ec69c4ca None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.105506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762eea2-cc74-4ed1-aea4-dbc8ec69c4ca None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.105506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762eea2-cc74-4ed1-aea4-dbc8ec69c4ca None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.107036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 500/1028] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:23.112107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef31b-e959-4120-ab4b-09736493195c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.112505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef31b-e959-4120-ab4b-09736493195c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.112676 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef31b-e959-4120-ab4b-09736493195c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.130032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d6bef31b-e959-4120-ab4b-09736493195c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:23.131218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d6bef31b-e959-4120-ab4b-09736493195c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:23.170757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40ce60-5faa-48ac-8f26-f8a86446ec2c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.171087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40ce60-5faa-48ac-8f26-f8a86446ec2c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.171305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40ce60-5faa-48ac-8f26-f8a86446ec2c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.172669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 501/1029] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:17:23.177589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b25f92-7512-43d2-897a-3d725e6122e0 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.177873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b25f92-7512-43d2-897a-3d725e6122e0 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.178146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b25f92-7512-43d2-897a-3d725e6122e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.196699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-81b25f92-7512-43d2-897a-3d725e6122e0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:23.198681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-81b25f92-7512-43d2-897a-3d725e6122e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:23.205014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 529/1030] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:23.224525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef31b-e959-4120-ab4b-09736493195c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:23.263829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:23.277441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 502/1031] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:23.293718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b25f92-7512-43d2-897a-3d725e6122e0 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:23.333571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef31b-e959-4120-ab4b-09736493195c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:23.373961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:23.381955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f5126-926f-46a4-8afe-3272628b4c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:23.382776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f5126-926f-46a4-8afe-3272628b4c4f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.383061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f5126-926f-46a4-8afe-3272628b4c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.383339 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f5126-926f-46a4-8afe-3272628b4c4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.395232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f5126-926f-46a4-8afe-3272628b4c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-b97f5126-926f-46a4-8afe-3272628b4c4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:23.404190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:23.405165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:23.406016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:23.407292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b25f92-7512-43d2-897a-3d725e6122e0 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:23.410336 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 530/1032] 10.222.0.30 () {56 vars in 1120 bytes} [Mon May 23 15:17:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:23.411206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f5126-926f-46a4-8afe-3272628b4c4f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:23.443776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af92c64-c854-43b6-8cf7-f41aeee7539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:23.444563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af92c64-c854-43b6-8cf7-f41aeee7539c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.444841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af92c64-c854-43b6-8cf7-f41aeee7539c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.445137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af92c64-c854-43b6-8cf7-f41aeee7539c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.446305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af92c64-c854-43b6-8cf7-f41aeee7539c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:23.448297 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af92c64-c854-43b6-8cf7-f41aeee7539c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:23.449945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 503/1033] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:23.455874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af92c64-c854-43b6-8cf7-f41aeee7539c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:23.517336 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f5126-926f-46a4-8afe-3272628b4c4f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:23.530341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:23.554693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761adee-d18a-4c56-a25e-fee637139514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:23.555516 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761adee-d18a-4c56-a25e-fee637139514 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.555806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761adee-d18a-4c56-a25e-fee637139514 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.556084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761adee-d18a-4c56-a25e-fee637139514 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.557352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761adee-d18a-4c56-a25e-fee637139514 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:23.557685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761adee-d18a-4c56-a25e-fee637139514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:23.559050 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761adee-d18a-4c56-a25e-fee637139514 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:23.564521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af92c64-c854-43b6-8cf7-f41aeee7539c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:23.567044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 531/1034] 10.222.0.30 () {56 vars in 1115 bytes} [Mon May 23 15:17:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:23.568339 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc677e5c-f933-4f83-a18f-65c224a6c154 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.568634 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc677e5c-f933-4f83-a18f-65c224a6c154 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.568959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc677e5c-f933-4f83-a18f-65c224a6c154 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.570342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 532/1035] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:23.573584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc677e5c-f933-4f83-a18f-65c224a6c154 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:23.598875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04629344-83b6-49cf-9d41-e71e8286bef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:23.599551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04629344-83b6-49cf-9d41-e71e8286bef8 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.599834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04629344-83b6-49cf-9d41-e71e8286bef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.600111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04629344-83b6-49cf-9d41-e71e8286bef8 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.600816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04629344-83b6-49cf-9d41-e71e8286bef8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:23.602295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04629344-83b6-49cf-9d41-e71e8286bef8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:23.622662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 504/1036] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:17:23 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:23.623928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043a170-d583-4581-8641-9a23bdf5945a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.624279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043a170-d583-4581-8641-9a23bdf5945a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.624493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043a170-d583-4581-8641-9a23bdf5945a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.631412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:23.642933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-3043a170-d583-4581-8641-9a23bdf5945a None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:23.644165 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3043a170-d583-4581-8641-9a23bdf5945a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:23.692170 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc677e5c-f933-4f83-a18f-65c224a6c154 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:23.723240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 505/1037] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:23.731443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26929033-5f52-4459-8d5b-cc78c57a32cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:23.732347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26929033-5f52-4459-8d5b-cc78c57a32cd None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.732633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26929033-5f52-4459-8d5b-cc78c57a32cd None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.732940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26929033-5f52-4459-8d5b-cc78c57a32cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.745134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26929033-5f52-4459-8d5b-cc78c57a32cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:23.747068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26929033-5f52-4459-8d5b-cc78c57a32cd None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:23.747938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043a170-d583-4581-8641-9a23bdf5945a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-26929033-5f52-4459-8d5b-cc78c57a32cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:23.769272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:23.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:23.771453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:23.771453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:23.771453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:23.771453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:23.771453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 533/1038] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:23.777137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26929033-5f52-4459-8d5b-cc78c57a32cd None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:23.823073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:23.878451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043a170-d583-4581-8641-9a23bdf5945a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:23.898272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26929033-5f52-4459-8d5b-cc78c57a32cd None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:23.901402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:23.920470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b0bac-384f-4f71-96de-8464a77507df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:23.921346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b0bac-384f-4f71-96de-8464a77507df None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.921631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b0bac-384f-4f71-96de-8464a77507df None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.921908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b0bac-384f-4f71-96de-8464a77507df None admin] PATH_INFO: `/v3/services/image` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.922573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b0bac-384f-4f71-96de-8464a77507df None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:23.923988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b0bac-384f-4f71-96de-8464a77507df None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-794b0bac-384f-4f71-96de-8464a77507df None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:23.935847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:23.936854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:23.937708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 506/1039] 10.222.0.30 () {56 vars in 1105 bytes} [Mon May 23 15:17:23 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:23.939511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42043f8-7514-42ae-adeb-068f25d5d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:23.940178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42043f8-7514-42ae-adeb-068f25d5d481 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:23.940483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42043f8-7514-42ae-adeb-068f25d5d481 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:23.940757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42043f8-7514-42ae-adeb-068f25d5d481 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:23.941511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42043f8-7514-42ae-adeb-068f25d5d481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:23.941816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42043f8-7514-42ae-adeb-068f25d5d481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:23.943004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b0bac-384f-4f71-96de-8464a77507df None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:23.943446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42043f8-7514-42ae-adeb-068f25d5d481 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:23.956696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 534/1040] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:23.962295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42043f8-7514-42ae-adeb-068f25d5d481 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:24.056475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b0bac-384f-4f71-96de-8464a77507df None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:24.066293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42043f8-7514-42ae-adeb-068f25d5d481 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:24.074830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:24.094635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d50953-ae89-43f6-9568-a58def0cae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:24.095314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d50953-ae89-43f6-9568-a58def0cae10 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:24.095603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d50953-ae89-43f6-9568-a58def0cae10 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:24.095885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d50953-ae89-43f6-9568-a58def0cae10 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:24.096600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d50953-ae89-43f6-9568-a58def0cae10 None admin] RBAC: Adding query filter params (name=image) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:24.096938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d50953-ae89-43f6-9568-a58def0cae10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:24.098318 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d50953-ae89-43f6-9568-a58def0cae10 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:24.108549 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 507/1041] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:24.109150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d827924-8834-4b0b-b54a-4bce857e8216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:24.109150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d827924-8834-4b0b-b54a-4bce857e8216 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:24.109150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d827924-8834-4b0b-b54a-4bce857e8216 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:24.109150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d827924-8834-4b0b-b54a-4bce857e8216 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:24.109588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d50953-ae89-43f6-9568-a58def0cae10 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:24.117193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d827924-8834-4b0b-b54a-4bce857e8216 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:24.119188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d827924-8834-4b0b-b54a-4bce857e8216 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-7d827924-8834-4b0b-b54a-4bce857e8216 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:17:24.137056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:24.138250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 535/1042] 10.222.0.30 () {56 vars in 1093 bytes} [Mon May 23 15:17:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:24.144101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d827924-8834-4b0b-b54a-4bce857e8216 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:24.182161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:24.233758 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d50953-ae89-43f6-9568-a58def0cae10 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:24.265889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d827924-8834-4b0b-b54a-4bce857e8216 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:24.271133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc41575-1641-4bcc-b361-4d8ae80136ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:24.271802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc41575-1641-4bcc-b361-4d8ae80136ef None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:24.272085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc41575-1641-4bcc-b361-4d8ae80136ef None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:24.272361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc41575-1641-4bcc-b361-4d8ae80136ef None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:24.273096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc41575-1641-4bcc-b361-4d8ae80136ef None admin] RBAC: Adding query filter params (type=image) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:24.273696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc41575-1641-4bcc-b361-4d8ae80136ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:24.274780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc41575-1641-4bcc-b361-4d8ae80136ef None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:24.280741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 508/1043] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:24.285956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc41575-1641-4bcc-b361-4d8ae80136ef None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:24.303575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2980146-8964-4ddb-bf4b-53aa3201e7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:24.304354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2980146-8964-4ddb-bf4b-53aa3201e7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:24.304731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2980146-8964-4ddb-bf4b-53aa3201e7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:24.305052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2980146-8964-4ddb-bf4b-53aa3201e7c6 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:24.305884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2980146-8964-4ddb-bf4b-53aa3201e7c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:24.306192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2980146-8964-4ddb-bf4b-53aa3201e7c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:24.308168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2980146-8964-4ddb-bf4b-53aa3201e7c6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:24.329206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 536/1044] 10.222.0.30 () {56 vars in 1133 bytes} [Mon May 23 15:17:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:24.332230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2980146-8964-4ddb-bf4b-53aa3201e7c6 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:24.335307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:24.397462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc41575-1641-4bcc-b361-4d8ae80136ef None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:24.424395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:24.434676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab40d5f-926a-41f2-9273-68631b380aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:24.435448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab40d5f-926a-41f2-9273-68631b380aba None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:24.435731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab40d5f-926a-41f2-9273-68631b380aba None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:24.436017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab40d5f-926a-41f2-9273-68631b380aba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:24.436785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab40d5f-926a-41f2-9273-68631b380aba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:24.438207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab40d5f-926a-41f2-9273-68631b380aba None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:24.445425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2980146-8964-4ddb-bf4b-53aa3201e7c6 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:24.468910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 509/1045] 10.222.0.30 () {60 vars in 1144 bytes} [Mon May 23 15:17:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:17:24.506931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1b791-a207-4906-9c72-accb7ce46184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:24.507711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b791-a207-4906-9c72-accb7ce46184 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:24.508058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b791-a207-4906-9c72-accb7ce46184 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:24.508397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b791-a207-4906-9c72-accb7ce46184 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:24.592918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b791-a207-4906-9c72-accb7ce46184 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:24.592918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b791-a207-4906-9c72-accb7ce46184 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:24.677748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:24.703746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 537/1046] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:17:24.709549 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab40d5f-926a-41f2-9273-68631b380aba None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:24.814486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab40d5f-926a-41f2-9273-68631b380aba None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:24.816349 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:24.859555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec1852-f106-4875-9f05-df5d0861041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:24.860661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec1852-f106-4875-9f05-df5d0861041e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:24.861159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec1852-f106-4875-9f05-df5d0861041e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:24.861620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec1852-f106-4875-9f05-df5d0861041e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:24.873119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec1852-f106-4875-9f05-df5d0861041e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:24.874851 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec1852-f106-4875-9f05-df5d0861041e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-31ec1852-f106-4875-9f05-df5d0861041e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:24.893356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:24.894257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:24.895147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:24.895147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:24.895147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:24.895147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:24.896839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 510/1047] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:24.902870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1b791-a207-4906-9c72-accb7ce46184 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:25.004647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1b791-a207-4906-9c72-accb7ce46184 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:25.038649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb0f21-afb7-4c9a-969b-44957d40aeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:25.039320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb0f21-afb7-4c9a-969b-44957d40aeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:25.039602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb0f21-afb7-4c9a-969b-44957d40aeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:25.039877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb0f21-afb7-4c9a-969b-44957d40aeb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:25.040590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb0f21-afb7-4c9a-969b-44957d40aeb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:25.040915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb0f21-afb7-4c9a-969b-44957d40aeb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:25.042819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb0f21-afb7-4c9a-969b-44957d40aeb4 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:25.059247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 538/1048] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:25.060475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec1852-f106-4875-9f05-df5d0861041e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:25.167051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec1852-f106-4875-9f05-df5d0861041e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:25.202313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edf5d0-a801-4ce9-8eed-406335600b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:25.203108 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edf5d0-a801-4ce9-8eed-406335600b1f None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:25.203392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edf5d0-a801-4ce9-8eed-406335600b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:25.203664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edf5d0-a801-4ce9-8eed-406335600b1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:25.210596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edf5d0-a801-4ce9-8eed-406335600b1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:25.212038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edf5d0-a801-4ce9-8eed-406335600b1f None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-59edf5d0-a801-4ce9-8eed-406335600b1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:17:25.226844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:25.227763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:25.228631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:17:25.228631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:17:25.228631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:17:25.228631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:17:25.228631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:17:25.228631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:17:25.228631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:25.228631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:25.233040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 511/1049] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:25.234793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb0f21-afb7-4c9a-969b-44957d40aeb4 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:25.271258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:25.340179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb0f21-afb7-4c9a-969b-44957d40aeb4 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:25.374297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413003d-5734-4812-9760-fc56c9ab136a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:25.374961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413003d-5734-4812-9760-fc56c9ab136a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:25.375242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413003d-5734-4812-9760-fc56c9ab136a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:25.375516 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413003d-5734-4812-9760-fc56c9ab136a None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:25.376551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413003d-5734-4812-9760-fc56c9ab136a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:25.376858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413003d-5734-4812-9760-fc56c9ab136a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:25.378306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413003d-5734-4812-9760-fc56c9ab136a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:25.395289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 539/1050] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:25.396538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edf5d0-a801-4ce9-8eed-406335600b1f None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:25.413330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:25.502201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edf5d0-a801-4ce9-8eed-406335600b1f None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:25.537986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c049ef6-3625-408e-8012-2acc170cf803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:25.538563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c049ef6-3625-408e-8012-2acc170cf803 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:25.538845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c049ef6-3625-408e-8012-2acc170cf803 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:25.539120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c049ef6-3625-408e-8012-2acc170cf803 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:25.539947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c049ef6-3625-408e-8012-2acc170cf803 None admin] RBAC: Adding query filter params (role.id=e3f9cda0e462469f92d47a2624887c97, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=232cfaefaf584ac59720713e02cf2c2d) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:25.540260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c049ef6-3625-408e-8012-2acc170cf803 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:25.541924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c049ef6-3625-408e-8012-2acc170cf803 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:25.553161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 512/1051] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:17:25 2022] GET /identity/v3/role_assignments?user.id=232cfaefaf584ac59720713e02cf2c2d&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=e3f9cda0e462469f92d47a2624887c97 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:25.939684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083185-4f3c-450c-853c-adfb3c257fbf None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:25.940028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083185-4f3c-450c-853c-adfb3c257fbf None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:25.940144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083185-4f3c-450c-853c-adfb3c257fbf None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:25.941715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 540/1052] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:25.946674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cc5de-0507-4e90-b37f-33a364965fb9 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:25.946961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cc5de-0507-4e90-b37f-33a364965fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:25.947234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cc5de-0507-4e90-b37f-33a364965fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:25.964529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ca4cc5de-0507-4e90-b37f-33a364965fb9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:25.965719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ca4cc5de-0507-4e90-b37f-33a364965fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:26.034553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 513/1053] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:26.057746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70083185-4f3c-450c-853c-adfb3c257fbf None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:26.157517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:26.165907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70083185-4f3c-450c-853c-adfb3c257fbf None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:26.200515 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b11c8-3c17-45a2-82cb-b0484906f31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:26.201257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b11c8-3c17-45a2-82cb-b0484906f31c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:26.201538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b11c8-3c17-45a2-82cb-b0484906f31c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:26.201809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b11c8-3c17-45a2-82cb-b0484906f31c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:26.211859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b11c8-3c17-45a2-82cb-b0484906f31c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-ea3b11c8-3c17-45a2-82cb-b0484906f31c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:26.219806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:26.220753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:26.221699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:26.225221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 541/1054] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:17:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:26.226947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cc5de-0507-4e90-b37f-33a364965fb9 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:26.238008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:26.330568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cc5de-0507-4e90-b37f-33a364965fb9 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:26.366328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f454b-6582-4bdf-b8ca-0c4da9d7bb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:26.367000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f454b-6582-4bdf-b8ca-0c4da9d7bb46 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:26.367279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f454b-6582-4bdf-b8ca-0c4da9d7bb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:26.367548 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f454b-6582-4bdf-b8ca-0c4da9d7bb46 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:26.368295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f454b-6582-4bdf-b8ca-0c4da9d7bb46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:26.368598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f454b-6582-4bdf-b8ca-0c4da9d7bb46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:26.370287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f454b-6582-4bdf-b8ca-0c4da9d7bb46 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:26.379643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 514/1055] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:17:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:26.386021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b11c8-3c17-45a2-82cb-b0484906f31c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:26.487854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b11c8-3c17-45a2-82cb-b0484906f31c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:26.521283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb96fa-27a6-4d80-aa67-e817bb5c1e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:26.522157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb96fa-27a6-4d80-aa67-e817bb5c1e8c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:26.522440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb96fa-27a6-4d80-aa67-e817bb5c1e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:26.522713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb96fa-27a6-4d80-aa67-e817bb5c1e8c None admin] PATH_INFO: `/v3/users/16eac8308bac43ddb96fb5521c145dd1` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:26.537015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb96fa-27a6-4d80-aa67-e817bb5c1e8c None admin] RBAC: Authorizing `identity:get_user(user_id=16eac8308bac43ddb96fb5521c145dd1)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:26.538514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb96fa-27a6-4d80-aa67-e817bb5c1e8c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:26.545207 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 542/1056] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:26 2022] GET /identity/v3/users/16eac8308bac43ddb96fb5521c145dd1 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:26.546348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f454b-6582-4bdf-b8ca-0c4da9d7bb46 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:26.626858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:26.662157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f454b-6582-4bdf-b8ca-0c4da9d7bb46 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:26.697966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f045525-fa4f-4afc-8c4d-1bb7f3e15070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:26.698611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f045525-fa4f-4afc-8c4d-1bb7f3e15070 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:26.698899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f045525-fa4f-4afc-8c4d-1bb7f3e15070 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:26.699176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f045525-fa4f-4afc-8c4d-1bb7f3e15070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:26.751447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986eb74-e864-4cfb-8cc8-2e58799414c7 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:26.751648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986eb74-e864-4cfb-8cc8-2e58799414c7 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:26.751920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986eb74-e864-4cfb-8cc8-2e58799414c7 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:26.753336 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 543/1057] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:17:26.758159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e481-8fbc-4f16-8dc4-99485b6a35da None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:26.758457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e481-8fbc-4f16-8dc4-99485b6a35da None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:26.758728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e481-8fbc-4f16-8dc4-99485b6a35da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:26.774035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f045525-fa4f-4afc-8c4d-1bb7f3e15070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:26.775509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f045525-fa4f-4afc-8c4d-1bb7f3e15070 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:26.777237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-9e09e481-8fbc-4f16-8dc4-99485b6a35da None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:26.778416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9e09e481-8fbc-4f16-8dc4-99485b6a35da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:26.788123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:26.848770 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:26.852290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 544/1058] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:26.874606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09e481-8fbc-4f16-8dc4-99485b6a35da None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:26.888592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 515/1059] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:17:26.889624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f045525-fa4f-4afc-8c4d-1bb7f3e15070 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:26.982911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09e481-8fbc-4f16-8dc4-99485b6a35da None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:26.990655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f045525-fa4f-4afc-8c4d-1bb7f3e15070 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:27.019166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c34e1c-7593-4bd8-8411-2ccc47f3671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:27.019828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c34e1c-7593-4bd8-8411-2ccc47f3671c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:27.020110 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c34e1c-7593-4bd8-8411-2ccc47f3671c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:27.020497 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c34e1c-7593-4bd8-8411-2ccc47f3671c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:27.029316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171d31d-1f23-45cf-b195-a4eb9fb273f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:27.030195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171d31d-1f23-45cf-b195-a4eb9fb273f6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:27.030457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171d31d-1f23-45cf-b195-a4eb9fb273f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:27.030726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171d31d-1f23-45cf-b195-a4eb9fb273f6 None admin] PATH_INFO: `/v3/projects/a407a0bd0f334e1aac397d76ada3a767` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:27.032256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171d31d-1f23-45cf-b195-a4eb9fb273f6 None admin] RBAC: Authorizing `identity:get_project(project_id=a407a0bd0f334e1aac397d76ada3a767)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:27.034081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171d31d-1f23-45cf-b195-a4eb9fb273f6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:27.035288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:27.041000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 516/1060] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:17:26 2022] GET /identity/v3/projects/a407a0bd0f334e1aac397d76ada3a767 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:27.041740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171d31d-1f23-45cf-b195-a4eb9fb273f6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:27.084248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:27.098644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c34e1c-7593-4bd8-8411-2ccc47f3671c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:27.100115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c34e1c-7593-4bd8-8411-2ccc47f3671c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:27.155412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171d31d-1f23-45cf-b195-a4eb9fb273f6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:27.191171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a52f8-4275-42d2-bf09-65bb1d888ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:27.191829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a52f8-4275-42d2-bf09-65bb1d888ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:27.192111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a52f8-4275-42d2-bf09-65bb1d888ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:27.192385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a52f8-4275-42d2-bf09-65bb1d888ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:27.193259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a52f8-4275-42d2-bf09-65bb1d888ae9 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=a407a0bd0f334e1aac397d76ada3a767, user.id=16eac8308bac43ddb96fb5521c145dd1) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:27.193569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a52f8-4275-42d2-bf09-65bb1d888ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:27.195124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a52f8-4275-42d2-bf09-65bb1d888ae9 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:27.201274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 517/1061] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:17:27 2022] GET /identity/v3/role_assignments?user.id=16eac8308bac43ddb96fb5521c145dd1&scope.project.id=a407a0bd0f334e1aac397d76ada3a767&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:27.216679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 545/1062] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:17:27.223187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a52f8-4275-42d2-bf09-65bb1d888ae9 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:27.333808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a52f8-4275-42d2-bf09-65bb1d888ae9 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:27.335421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:27.374020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91137c-ba34-4e2e-81ef-96bee0e75a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:27.374831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91137c-ba34-4e2e-81ef-96bee0e75a72 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:27.375113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91137c-ba34-4e2e-81ef-96bee0e75a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:27.375388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91137c-ba34-4e2e-81ef-96bee0e75a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:27.387859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91137c-ba34-4e2e-81ef-96bee0e75a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:27.388793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91137c-ba34-4e2e-81ef-96bee0e75a72 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-ca91137c-ba34-4e2e-81ef-96bee0e75a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:27.407486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:27.408474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:27.409913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:27.409913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:27.409913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:27.409913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:27.409913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 518/1063] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:27.414598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c34e1c-7593-4bd8-8411-2ccc47f3671c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:27.464092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:27.532461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c34e1c-7593-4bd8-8411-2ccc47f3671c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:27.566928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2aab35-215a-4856-8891-7b13b8628666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:27.567600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2aab35-215a-4856-8891-7b13b8628666 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:27.567886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2aab35-215a-4856-8891-7b13b8628666 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:27.568163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2aab35-215a-4856-8891-7b13b8628666 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:27.568910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2aab35-215a-4856-8891-7b13b8628666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:27.569281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2aab35-215a-4856-8891-7b13b8628666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:27.570639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2aab35-215a-4856-8891-7b13b8628666 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:27.582213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 546/1064] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:27.825976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923022e-d94a-4859-ab58-9120ff067d29 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:27.826264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923022e-d94a-4859-ab58-9120ff067d29 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:27.826544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923022e-d94a-4859-ab58-9120ff067d29 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:27.827946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 519/1065] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:27.833366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8233235-a123-46e7-9fbf-122e0a0c1c38 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:27.833663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8233235-a123-46e7-9fbf-122e0a0c1c38 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:27.833949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8233235-a123-46e7-9fbf-122e0a0c1c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:27.851258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a8233235-a123-46e7-9fbf-122e0a0c1c38 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:27.852475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a8233235-a123-46e7-9fbf-122e0a0c1c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:27.923446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 547/1066] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:27.946542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923022e-d94a-4859-ab58-9120ff067d29 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:28.047782 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923022e-d94a-4859-ab58-9120ff067d29 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:28.080783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af9fcf-9ff1-4e79-9555-9a20c0ca8da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:28.081609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af9fcf-9ff1-4e79-9555-9a20c0ca8da7 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:28.081891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af9fcf-9ff1-4e79-9555-9a20c0ca8da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:28.082164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af9fcf-9ff1-4e79-9555-9a20c0ca8da7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:28.092638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af9fcf-9ff1-4e79-9555-9a20c0ca8da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-53af9fcf-9ff1-4e79-9555-9a20c0ca8da7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:28.100735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:28.101857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:28.108358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 520/1067] 10.222.0.30 () {56 vars in 1120 bytes} [Mon May 23 15:17:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:28.110626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8233235-a123-46e7-9fbf-122e0a0c1c38 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:28.143384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:28.207334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8233235-a123-46e7-9fbf-122e0a0c1c38 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:28.241728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3864981-a4f4-415a-a42c-a2af773c8bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:28.242392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3864981-a4f4-415a-a42c-a2af773c8bfd None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:28.242671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3864981-a4f4-415a-a42c-a2af773c8bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:28.242942 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3864981-a4f4-415a-a42c-a2af773c8bfd None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:28.243714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3864981-a4f4-415a-a42c-a2af773c8bfd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:28.244020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3864981-a4f4-415a-a42c-a2af773c8bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:28.245442 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3864981-a4f4-415a-a42c-a2af773c8bfd None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:28.253619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 548/1068] 10.222.0.30 () {56 vars in 1115 bytes} [Mon May 23 15:17:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:28.259440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af9fcf-9ff1-4e79-9555-9a20c0ca8da7 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:28.275471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:28.362872 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af9fcf-9ff1-4e79-9555-9a20c0ca8da7 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:28.401580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93213a-3f33-42e8-82e3-0b56af889da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:28.402382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93213a-3f33-42e8-82e3-0b56af889da1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:28.402666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93213a-3f33-42e8-82e3-0b56af889da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:28.402940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93213a-3f33-42e8-82e3-0b56af889da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:28.413521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93213a-3f33-42e8-82e3-0b56af889da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:28.415004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93213a-3f33-42e8-82e3-0b56af889da1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-de93213a-3f33-42e8-82e3-0b56af889da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:28.432672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:28.433746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:28.434686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:28.434686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:28.434686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:28.434686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:28.434686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 521/1069] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:28.439704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3864981-a4f4-415a-a42c-a2af773c8bfd None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:28.532647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:28.538561 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3864981-a4f4-415a-a42c-a2af773c8bfd None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:28.570527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860915d-8fb8-45c1-bf52-9694b166df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:28.571179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860915d-8fb8-45c1-bf52-9694b166df14 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:28.571457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860915d-8fb8-45c1-bf52-9694b166df14 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:28.571730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860915d-8fb8-45c1-bf52-9694b166df14 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:28.572432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860915d-8fb8-45c1-bf52-9694b166df14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:28.573320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860915d-8fb8-45c1-bf52-9694b166df14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:28.575056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860915d-8fb8-45c1-bf52-9694b166df14 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:28.591325 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 549/1070] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:28.592924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93213a-3f33-42e8-82e3-0b56af889da1 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:28.658968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:28.694372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93213a-3f33-42e8-82e3-0b56af889da1 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:28.726788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5087b-0d50-46f7-a9ec-bf35e874e967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:28.727566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5087b-0d50-46f7-a9ec-bf35e874e967 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:28.727842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5087b-0d50-46f7-a9ec-bf35e874e967 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:28.728112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5087b-0d50-46f7-a9ec-bf35e874e967 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:28.736342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5087b-0d50-46f7-a9ec-bf35e874e967 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:28.737856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5087b-0d50-46f7-a9ec-bf35e874e967 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-49a5087b-0d50-46f7-a9ec-bf35e874e967 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:17:28.752501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:28.753701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:28.754691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 522/1071] 10.222.0.30 () {56 vars in 1093 bytes} [Mon May 23 15:17:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:28.759541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860915d-8fb8-45c1-bf52-9694b166df14 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:28.859451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860915d-8fb8-45c1-bf52-9694b166df14 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:28.892488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820424-c199-49ae-931d-b129e2fe4fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:28.893183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820424-c199-49ae-931d-b129e2fe4fdf None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:28.893462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820424-c199-49ae-931d-b129e2fe4fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:28.893732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820424-c199-49ae-931d-b129e2fe4fdf None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:28.894551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820424-c199-49ae-931d-b129e2fe4fdf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:28.894855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820424-c199-49ae-931d-b129e2fe4fdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:28.896266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820424-c199-49ae-931d-b129e2fe4fdf None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:28.913949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 550/1072] 10.222.0.30 () {56 vars in 1133 bytes} [Mon May 23 15:17:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:28.920143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5087b-0d50-46f7-a9ec-bf35e874e967 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:29.018336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5087b-0d50-46f7-a9ec-bf35e874e967 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:29.043884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:29.053178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b97d9d-9fc3-44ab-87fc-58a8d9ce3d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:29.053977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97d9d-9fc3-44ab-87fc-58a8d9ce3d3b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.054262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97d9d-9fc3-44ab-87fc-58a8d9ce3d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.054539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97d9d-9fc3-44ab-87fc-58a8d9ce3d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.065287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b97d9d-9fc3-44ab-87fc-58a8d9ce3d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:29.066729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b97d9d-9fc3-44ab-87fc-58a8d9ce3d3b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-a2b97d9d-9fc3-44ab-87fc-58a8d9ce3d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:29.083579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:29.084529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:29.085624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:29.085624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:29.085624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:29.085624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:29.085624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 523/1073] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:29.091212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820424-c199-49ae-931d-b129e2fe4fdf None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:29.107198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:29.189513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820424-c199-49ae-931d-b129e2fe4fdf None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:29.227447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e890615-3c1d-4fa4-8fc4-73f377be042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:29.228121 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e890615-3c1d-4fa4-8fc4-73f377be042c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.228406 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e890615-3c1d-4fa4-8fc4-73f377be042c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.228684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e890615-3c1d-4fa4-8fc4-73f377be042c None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.229588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e890615-3c1d-4fa4-8fc4-73f377be042c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:29.229869 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e890615-3c1d-4fa4-8fc4-73f377be042c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:29.231226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e890615-3c1d-4fa4-8fc4-73f377be042c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:29.242801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 551/1074] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:29.248325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b97d9d-9fc3-44ab-87fc-58a8d9ce3d3b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:29.346491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b97d9d-9fc3-44ab-87fc-58a8d9ce3d3b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:29.379030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d216002-913a-4ee4-b394-075530d2ae19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:29.379835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d216002-913a-4ee4-b394-075530d2ae19 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.380115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d216002-913a-4ee4-b394-075530d2ae19 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.380388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d216002-913a-4ee4-b394-075530d2ae19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.380647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa1db2-aa09-40ac-a709-8470692dda64 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.380973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa1db2-aa09-40ac-a709-8470692dda64 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.381251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa1db2-aa09-40ac-a709-8470692dda64 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.382624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 552/1075] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:29.388258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caf0ba-fcaa-41b3-a40e-a15bdd6f29e9 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.388541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caf0ba-fcaa-41b3-a40e-a15bdd6f29e9 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.388824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caf0ba-fcaa-41b3-a40e-a15bdd6f29e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.389499 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d216002-913a-4ee4-b394-075530d2ae19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:29.390950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d216002-913a-4ee4-b394-075530d2ae19 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:29.405841 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b5caf0ba-fcaa-41b3-a40e-a15bdd6f29e9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-1d216002-913a-4ee4-b394-075530d2ae19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:17:29.406046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:29.407034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:29.407910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:17:29.407910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:17:29.407910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:17:29.407910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:17:29.407910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:17:29.407910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:17:29.407910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:29.407910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:29.408441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b5caf0ba-fcaa-41b3-a40e-a15bdd6f29e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:29.409172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 524/1076] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:29.412845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d216002-913a-4ee4-b394-075530d2ae19 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:29.452453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:29.474398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 553/1077] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:29.497242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caf0ba-fcaa-41b3-a40e-a15bdd6f29e9 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:29.500100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:29.513969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d216002-913a-4ee4-b394-075530d2ae19 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:29.547879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b79f5-8da2-4abc-aea6-48f30e0aa1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:29.548533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b79f5-8da2-4abc-aea6-48f30e0aa1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.548817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b79f5-8da2-4abc-aea6-48f30e0aa1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.549125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b79f5-8da2-4abc-aea6-48f30e0aa1b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.549908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b79f5-8da2-4abc-aea6-48f30e0aa1b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:29.550218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b79f5-8da2-4abc-aea6-48f30e0aa1b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:29.551644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b79f5-8da2-4abc-aea6-48f30e0aa1b8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:29.563271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 525/1078] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:29.569050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b79f5-8da2-4abc-aea6-48f30e0aa1b8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:29.597335 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caf0ba-fcaa-41b3-a40e-a15bdd6f29e9 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:29.634082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748cd71-ac01-4ee7-aad8-f6a0e63f0da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:29.634871 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748cd71-ac01-4ee7-aad8-f6a0e63f0da3 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.635150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748cd71-ac01-4ee7-aad8-f6a0e63f0da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.635478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748cd71-ac01-4ee7-aad8-f6a0e63f0da3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.646009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748cd71-ac01-4ee7-aad8-f6a0e63f0da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-c748cd71-ac01-4ee7-aad8-f6a0e63f0da3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:29.654215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:29.655169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:29.656292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 554/1079] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:29.661210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748cd71-ac01-4ee7-aad8-f6a0e63f0da3 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:29.673107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:29.677167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b79f5-8da2-4abc-aea6-48f30e0aa1b8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:29.711295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a804e0-bb4f-4819-aff4-380956b0e3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:29.712106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a804e0-bb4f-4819-aff4-380956b0e3fb None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.712382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a804e0-bb4f-4819-aff4-380956b0e3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.712649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a804e0-bb4f-4819-aff4-380956b0e3fb None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/232cfaefaf584ac59720713e02cf2c2d/roles/e3f9cda0e462469f92d47a2624887c97` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.715570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a804e0-bb4f-4819-aff4-380956b0e3fb None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=232cfaefaf584ac59720713e02cf2c2d, role_id=e3f9cda0e462469f92d47a2624887c97)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:29.717046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a804e0-bb4f-4819-aff4-380956b0e3fb None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:29.727206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:29.730524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 526/1080] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:17:29 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/232cfaefaf584ac59720713e02cf2c2d/roles/e3f9cda0e462469f92d47a2624887c97 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:17:29.770126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748cd71-ac01-4ee7-aad8-f6a0e63f0da3 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:29.805323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c9170-e030-42a1-9dd4-836ca0b7fe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:29.806011 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c9170-e030-42a1-9dd4-836ca0b7fe28 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.806295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c9170-e030-42a1-9dd4-836ca0b7fe28 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.806569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c9170-e030-42a1-9dd4-836ca0b7fe28 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.807866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c9170-e030-42a1-9dd4-836ca0b7fe28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:29.808172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c9170-e030-42a1-9dd4-836ca0b7fe28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:29.809724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c9170-e030-42a1-9dd4-836ca0b7fe28 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:29.819299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 555/1081] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:29.825687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a804e0-bb4f-4819-aff4-380956b0e3fb None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:29.931456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a804e0-bb4f-4819-aff4-380956b0e3fb None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:29.965030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33e9d4-c0d4-4736-b3f0-d31e72bc7359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:29.965823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33e9d4-c0d4-4736-b3f0-d31e72bc7359 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:29.966099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33e9d4-c0d4-4736-b3f0-d31e72bc7359 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:29.966367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33e9d4-c0d4-4736-b3f0-d31e72bc7359 None admin] PATH_INFO: `/v3/users/16eac8308bac43ddb96fb5521c145dd1` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:29.967535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33e9d4-c0d4-4736-b3f0-d31e72bc7359 None admin] RBAC: Authorizing `identity:get_user(user_id=16eac8308bac43ddb96fb5521c145dd1)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:29.969002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33e9d4-c0d4-4736-b3f0-d31e72bc7359 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:29.970626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 527/1082] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:29 2022] GET /identity/v3/users/16eac8308bac43ddb96fb5521c145dd1 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:29.976456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c9170-e030-42a1-9dd4-836ca0b7fe28 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:30.074641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c9170-e030-42a1-9dd4-836ca0b7fe28 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:30.099536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:30.108862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793d99d-091b-4eed-bab5-385aa8b98bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:30.109743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793d99d-091b-4eed-bab5-385aa8b98bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:30.110026 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793d99d-091b-4eed-bab5-385aa8b98bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:30.110299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793d99d-091b-4eed-bab5-385aa8b98bc5 None admin] PATH_INFO: `/v3/projects/a407a0bd0f334e1aac397d76ada3a767` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:30.111471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793d99d-091b-4eed-bab5-385aa8b98bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=a407a0bd0f334e1aac397d76ada3a767)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:30.112913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793d99d-091b-4eed-bab5-385aa8b98bc5 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:30.114613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 556/1083] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:17:29 2022] GET /identity/v3/projects/a407a0bd0f334e1aac397d76ada3a767 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:30.120375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33e9d4-c0d4-4736-b3f0-d31e72bc7359 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:30.148127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:30.220889 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33e9d4-c0d4-4736-b3f0-d31e72bc7359 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:30.253166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb31dd2-224f-4e81-94b8-3cf1909fd5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:30.254050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb31dd2-224f-4e81-94b8-3cf1909fd5a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:30.254327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb31dd2-224f-4e81-94b8-3cf1909fd5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:30.254602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb31dd2-224f-4e81-94b8-3cf1909fd5a5 None admin] PATH_INFO: `/v3/projects/a407a0bd0f334e1aac397d76ada3a767/users/16eac8308bac43ddb96fb5521c145dd1/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:30.256518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb31dd2-224f-4e81-94b8-3cf1909fd5a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a407a0bd0f334e1aac397d76ada3a767, user_id=16eac8308bac43ddb96fb5521c145dd1, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:30.258049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb31dd2-224f-4e81-94b8-3cf1909fd5a5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:30.271495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 528/1084] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:17:30 2022] PUT /identity/v3/projects/a407a0bd0f334e1aac397d76ada3a767/users/16eac8308bac43ddb96fb5521c145dd1/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:17:31.945318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6d1b2-884c-4a0d-8064-6b89152a48cd None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:31.945811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6d1b2-884c-4a0d-8064-6b89152a48cd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:31.945811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6d1b2-884c-4a0d-8064-6b89152a48cd None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:31.947180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 557/1085] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:31.953064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3fa35-fb51-41b2-9b07-672b92ad55fb None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:31.953463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3fa35-fb51-41b2-9b07-672b92ad55fb None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:31.953691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3fa35-fb51-41b2-9b07-672b92ad55fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:31.970223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4db3fa35-fb51-41b2-9b07-672b92ad55fb None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:31.971404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4db3fa35-fb51-41b2-9b07-672b92ad55fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:32.037597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 529/1086] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:32.061125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6d1b2-884c-4a0d-8064-6b89152a48cd None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:32.162858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6d1b2-884c-4a0d-8064-6b89152a48cd None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:32.196508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232380d6-f31f-4111-aa54-edc54198985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:32.197271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232380d6-f31f-4111-aa54-edc54198985e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:32.197555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232380d6-f31f-4111-aa54-edc54198985e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:32.197833 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232380d6-f31f-4111-aa54-edc54198985e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:32.208723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232380d6-f31f-4111-aa54-edc54198985e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-232380d6-f31f-4111-aa54-edc54198985e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:32.217632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:32.218572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:32.219403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 558/1087] 10.222.0.30 () {56 vars in 1120 bytes} [Mon May 23 15:17:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:32.224785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3fa35-fb51-41b2-9b07-672b92ad55fb None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:32.241951 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:32.327887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3fa35-fb51-41b2-9b07-672b92ad55fb None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:32.362703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a1305-e934-49e9-844b-6e2253066864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:32.363368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a1305-e934-49e9-844b-6e2253066864 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:32.363652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a1305-e934-49e9-844b-6e2253066864 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:32.363926 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a1305-e934-49e9-844b-6e2253066864 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:32.364680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a1305-e934-49e9-844b-6e2253066864 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:32.365032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a1305-e934-49e9-844b-6e2253066864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:32.366412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a1305-e934-49e9-844b-6e2253066864 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:32.374517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 530/1088] 10.222.0.30 () {56 vars in 1115 bytes} [Mon May 23 15:17:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:32.380305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232380d6-f31f-4111-aa54-edc54198985e None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:32.437043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:32.477584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851edc7-5018-497a-93ae-e8e0c57110e7 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:32.477801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851edc7-5018-497a-93ae-e8e0c57110e7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:32.478086 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851edc7-5018-497a-93ae-e8e0c57110e7 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:32.479530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 531/1089] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:17:32.481783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232380d6-f31f-4111-aa54-edc54198985e None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:32.484396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89e9c2-befc-43ed-ab34-268b93f9a089 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:32.484680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89e9c2-befc-43ed-ab34-268b93f9a089 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:32.484994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89e9c2-befc-43ed-ab34-268b93f9a089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:32.502162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8f89e9c2-befc-43ed-ab34-268b93f9a089 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:32.503342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8f89e9c2-befc-43ed-ab34-268b93f9a089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:32.516237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9332b-8b8f-47c0-9339-65eeaa5d4b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:32.517058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9332b-8b8f-47c0-9339-65eeaa5d4b68 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:32.517346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9332b-8b8f-47c0-9339-65eeaa5d4b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:32.517627 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9332b-8b8f-47c0-9339-65eeaa5d4b68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:32.529077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9332b-8b8f-47c0-9339-65eeaa5d4b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:32.530806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9332b-8b8f-47c0-9339-65eeaa5d4b68 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-17f9332b-8b8f-47c0-9339-65eeaa5d4b68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:32.549047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:32.549972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:32.551114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:32.551114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:32.551114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:32.551114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:32.551114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 559/1090] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:32.555819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9332b-8b8f-47c0-9339-65eeaa5d4b68 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:32.571772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 532/1091] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:32.595585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89e9c2-befc-43ed-ab34-268b93f9a089 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:32.606018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:32.661496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9332b-8b8f-47c0-9339-65eeaa5d4b68 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:32.668845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:32.696083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81569b5-2378-41ed-9bca-0fefc1edc897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:32.696741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81569b5-2378-41ed-9bca-0fefc1edc897 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:32.697063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81569b5-2378-41ed-9bca-0fefc1edc897 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:32.697340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81569b5-2378-41ed-9bca-0fefc1edc897 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:32.697432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89e9c2-befc-43ed-ab34-268b93f9a089 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:32.698073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81569b5-2378-41ed-9bca-0fefc1edc897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:32.698425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81569b5-2378-41ed-9bca-0fefc1edc897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:32.699891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81569b5-2378-41ed-9bca-0fefc1edc897 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:32.711395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 560/1092] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:32.717492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81569b5-2378-41ed-9bca-0fefc1edc897 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:32.732686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21a2b-f864-40bf-8b5e-d95068fa0cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:32.733438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21a2b-f864-40bf-8b5e-d95068fa0cfd None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:32.733722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21a2b-f864-40bf-8b5e-d95068fa0cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:32.733995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21a2b-f864-40bf-8b5e-d95068fa0cfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:32.744401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb21a2b-f864-40bf-8b5e-d95068fa0cfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-ddb21a2b-f864-40bf-8b5e-d95068fa0cfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:32.752726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:32.753880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:32.755019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 533/1093] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:32.759750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21a2b-f864-40bf-8b5e-d95068fa0cfd None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:32.814446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81569b5-2378-41ed-9bca-0fefc1edc897 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:32.825546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:32.849095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2f111-c495-4291-b79e-d8aaf212e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:32.849879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2f111-c495-4291-b79e-d8aaf212e7ea None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:32.850161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2f111-c495-4291-b79e-d8aaf212e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:32.850430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2f111-c495-4291-b79e-d8aaf212e7ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:32.857457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2f111-c495-4291-b79e-d8aaf212e7ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:32.858877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2f111-c495-4291-b79e-d8aaf212e7ea None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:32.859002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21a2b-f864-40bf-8b5e-d95068fa0cfd None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-bea2f111-c495-4291-b79e-d8aaf212e7ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:17:32.874367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:32.875362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:32.876514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 561/1094] 10.222.0.30 () {56 vars in 1093 bytes} [Mon May 23 15:17:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:32.881418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2f111-c495-4291-b79e-d8aaf212e7ea None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:32.894705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2757b-aa6e-4570-8bf7-9074759fa840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:32.895370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2757b-aa6e-4570-8bf7-9074759fa840 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:32.895653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2757b-aa6e-4570-8bf7-9074759fa840 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:32.895933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2757b-aa6e-4570-8bf7-9074759fa840 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:32.896676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2757b-aa6e-4570-8bf7-9074759fa840 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:32.897018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2757b-aa6e-4570-8bf7-9074759fa840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:32.898411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2757b-aa6e-4570-8bf7-9074759fa840 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:32.906592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 534/1095] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:17:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:32.911624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:32.912210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2757b-aa6e-4570-8bf7-9074759fa840 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:32.984349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2f111-c495-4291-b79e-d8aaf212e7ea None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:33.011497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2757b-aa6e-4570-8bf7-9074759fa840 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:33.018512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa2d6f-c299-411d-b046-769dbc73e997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:33.019176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2d6f-c299-411d-b046-769dbc73e997 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:33.019455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2d6f-c299-411d-b046-769dbc73e997 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:33.019729 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2d6f-c299-411d-b046-769dbc73e997 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:33.020548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2d6f-c299-411d-b046-769dbc73e997 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:33.020852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2d6f-c299-411d-b046-769dbc73e997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:33.022281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2d6f-c299-411d-b046-769dbc73e997 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:33.039192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:33.044005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 562/1096] 10.222.0.30 () {56 vars in 1133 bytes} [Mon May 23 15:17:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:33.044915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa2d6f-c299-411d-b046-769dbc73e997 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:33.045981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b29d9-9333-450b-9257-080915be209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:33.046884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b29d9-9333-450b-9257-080915be209d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:33.047391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b29d9-9333-450b-9257-080915be209d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:33.047537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b29d9-9333-450b-9257-080915be209d None admin] PATH_INFO: `/v3/users/16eac8308bac43ddb96fb5521c145dd1` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:33.049500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b29d9-9333-450b-9257-080915be209d None admin] RBAC: Authorizing `identity:get_user(user_id=16eac8308bac43ddb96fb5521c145dd1)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:33.050946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b29d9-9333-450b-9257-080915be209d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:33.057214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 535/1097] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:32 2022] GET /identity/v3/users/16eac8308bac43ddb96fb5521c145dd1 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:33.058153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b29d9-9333-450b-9257-080915be209d None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:33.125021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:33.142470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa2d6f-c299-411d-b046-769dbc73e997 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:33.158628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b29d9-9333-450b-9257-080915be209d None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:33.177521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be7a5f-3f96-411b-b38c-2a08a50c2732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:33.178179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be7a5f-3f96-411b-b38c-2a08a50c2732 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:33.178463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be7a5f-3f96-411b-b38c-2a08a50c2732 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:33.178737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be7a5f-3f96-411b-b38c-2a08a50c2732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:33.201405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104138c-8bd9-4182-afee-df078eaa4eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:33.202062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104138c-8bd9-4182-afee-df078eaa4eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:33.202347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104138c-8bd9-4182-afee-df078eaa4eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:33.202622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104138c-8bd9-4182-afee-df078eaa4eb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:33.247013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be7a5f-3f96-411b-b38c-2a08a50c2732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:33.248386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be7a5f-3f96-411b-b38c-2a08a50c2732 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:33.267244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:33.271510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104138c-8bd9-4182-afee-df078eaa4eb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:33.273008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104138c-8bd9-4182-afee-df078eaa4eb8 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:33.330259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:33.347401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 563/1098] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:17:33.353289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be7a5f-3f96-411b-b38c-2a08a50c2732 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:33.371936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 536/1099] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:17:33.377890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104138c-8bd9-4182-afee-df078eaa4eb8 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:33.449595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be7a5f-3f96-411b-b38c-2a08a50c2732 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:33.470698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:33.474610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104138c-8bd9-4182-afee-df078eaa4eb8 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:33.485431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702e4d9-667c-4627-aa82-0f057b184f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:33.486230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702e4d9-667c-4627-aa82-0f057b184f4c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:33.486517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702e4d9-667c-4627-aa82-0f057b184f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:33.486794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702e4d9-667c-4627-aa82-0f057b184f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:33.496776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702e4d9-667c-4627-aa82-0f057b184f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:33.498298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702e4d9-667c-4627-aa82-0f057b184f4c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:33.509594 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd8bb4-9770-4963-a405-38c3a1f06f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:33.510431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd8bb4-9770-4963-a405-38c3a1f06f95 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:33.510749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd8bb4-9770-4963-a405-38c3a1f06f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:33.511101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd8bb4-9770-4963-a405-38c3a1f06f95 None admin] PATH_INFO: `/v3/projects/a407a0bd0f334e1aac397d76ada3a767` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:33.512261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd8bb4-9770-4963-a405-38c3a1f06f95 None admin] RBAC: Authorizing `identity:get_project(project_id=a407a0bd0f334e1aac397d76ada3a767)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:33.513710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd8bb4-9770-4963-a405-38c3a1f06f95 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:33.515392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 537/1100] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:17:33 2022] GET /identity/v3/projects/a407a0bd0f334e1aac397d76ada3a767 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-7702e4d9-667c-4627-aa82-0f057b184f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:17:33.515716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:33.516749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:33.522543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:17:33.522543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:17:33.522543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:17:33.522543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:33.522543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 564/1101] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:17:33.522860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd8bb4-9770-4963-a405-38c3a1f06f95 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:33.522955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702e4d9-667c-4627-aa82-0f057b184f4c None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:33.567996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:33.627835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd8bb4-9770-4963-a405-38c3a1f06f95 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:33.643969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702e4d9-667c-4627-aa82-0f057b184f4c None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:33.663338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86b82b-1755-43f5-a196-23074e40a1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:33.663990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86b82b-1755-43f5-a196-23074e40a1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:33.664273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86b82b-1755-43f5-a196-23074e40a1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:33.664555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86b82b-1755-43f5-a196-23074e40a1f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:33.665411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86b82b-1755-43f5-a196-23074e40a1f5 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=a407a0bd0f334e1aac397d76ada3a767, user.id=16eac8308bac43ddb96fb5521c145dd1) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:33.665725 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86b82b-1755-43f5-a196-23074e40a1f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:33.667300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86b82b-1755-43f5-a196-23074e40a1f5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:33.673247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 538/1102] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:17:33 2022] GET /identity/v3/role_assignments?user.id=16eac8308bac43ddb96fb5521c145dd1&scope.project.id=a407a0bd0f334e1aac397d76ada3a767&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:33.678570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ef52d-759c-40d3-93a0-1ae0d497f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:33.679252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ef52d-759c-40d3-93a0-1ae0d497f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:33.679543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ef52d-759c-40d3-93a0-1ae0d497f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:33.679817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ef52d-759c-40d3-93a0-1ae0d497f8d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:33.680555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ef52d-759c-40d3-93a0-1ae0d497f8d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:33.680879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ef52d-759c-40d3-93a0-1ae0d497f8d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:33.682297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ef52d-759c-40d3-93a0-1ae0d497f8d8 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:33.695097 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 565/1103] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:33.701683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86b82b-1755-43f5-a196-23074e40a1f5 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:33.734400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:33.809010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86b82b-1755-43f5-a196-23074e40a1f5 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:33.848029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156e8df-61e5-417d-8a98-b7612c82535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:33.848839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156e8df-61e5-417d-8a98-b7612c82535b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:33.849145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156e8df-61e5-417d-8a98-b7612c82535b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:33.849417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156e8df-61e5-417d-8a98-b7612c82535b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:33.857775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156e8df-61e5-417d-8a98-b7612c82535b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:33.859228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156e8df-61e5-417d-8a98-b7612c82535b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-b156e8df-61e5-417d-8a98-b7612c82535b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:17:33.874443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:33.875361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:33.876438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:17:33.876438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:17:33.876438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:17:33.876438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:17:33.876438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:17:33.876438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:17:33.876438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:17:33.876438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:17:33.876438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 539/1104] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:17:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:33.881821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ef52d-759c-40d3-93a0-1ae0d497f8d8 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:33.978244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:33.984738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ef52d-759c-40d3-93a0-1ae0d497f8d8 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:34.017971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79c39b-b731-42fc-b678-b29bd1b7f71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:34.018635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79c39b-b731-42fc-b678-b29bd1b7f71b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:34.018916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79c39b-b731-42fc-b678-b29bd1b7f71b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:34.019193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79c39b-b731-42fc-b678-b29bd1b7f71b None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:34.019966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79c39b-b731-42fc-b678-b29bd1b7f71b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:34.020274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79c39b-b731-42fc-b678-b29bd1b7f71b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:34.021723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79c39b-b731-42fc-b678-b29bd1b7f71b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:34.034226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 566/1105] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:34.040143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156e8df-61e5-417d-8a98-b7612c82535b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:34.128531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:34.138665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156e8df-61e5-417d-8a98-b7612c82535b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:34.171524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc7c28-682f-4eb6-ab25-bbbc7b485690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:34.172172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc7c28-682f-4eb6-ab25-bbbc7b485690 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:34.172453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc7c28-682f-4eb6-ab25-bbbc7b485690 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:34.172727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc7c28-682f-4eb6-ab25-bbbc7b485690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:34.173583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc7c28-682f-4eb6-ab25-bbbc7b485690 None admin] RBAC: Adding query filter params (role.id=e3f9cda0e462469f92d47a2624887c97, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=232cfaefaf584ac59720713e02cf2c2d) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:34.173895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc7c28-682f-4eb6-ab25-bbbc7b485690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:34.175328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc7c28-682f-4eb6-ab25-bbbc7b485690 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:34.181342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 540/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:17:34 2022] GET /identity/v3/role_assignments?user.id=232cfaefaf584ac59720713e02cf2c2d&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=e3f9cda0e462469f92d47a2624887c97 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:35.909360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3264f-6061-41ec-8f2a-3c36f258444b None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:35.909880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3264f-6061-41ec-8f2a-3c36f258444b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:35.909880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3264f-6061-41ec-8f2a-3c36f258444b None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:35.911220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 567/1107] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:35.916305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6d0e4-9925-4b56-8ed6-a3adeaa1b7a8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:35.916675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6d0e4-9925-4b56-8ed6-a3adeaa1b7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:35.916846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6d0e4-9925-4b56-8ed6-a3adeaa1b7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:35.934307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-9cd6d0e4-9925-4b56-8ed6-a3adeaa1b7a8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:35.935464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-9cd6d0e4-9925-4b56-8ed6-a3adeaa1b7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:36.000322 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 541/1108] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:36.025158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3264f-6061-41ec-8f2a-3c36f258444b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:36.123557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3264f-6061-41ec-8f2a-3c36f258444b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:36.157205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89baf7-89e7-4d17-9c05-77c3cb076943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:36.157981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89baf7-89e7-4d17-9c05-77c3cb076943 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:36.158267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89baf7-89e7-4d17-9c05-77c3cb076943 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:36.158546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89baf7-89e7-4d17-9c05-77c3cb076943 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:36.159764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89baf7-89e7-4d17-9c05-77c3cb076943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:36.161165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89baf7-89e7-4d17-9c05-77c3cb076943 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:36.162755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 568/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:36.168228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6d0e4-9925-4b56-8ed6-a3adeaa1b7a8 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:36.248368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:36.264347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6d0e4-9925-4b56-8ed6-a3adeaa1b7a8 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:36.297143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26228b4-d735-4303-afdf-cc195b7aa18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:36.297799 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26228b4-d735-4303-afdf-cc195b7aa18a None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:36.298078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26228b4-d735-4303-afdf-cc195b7aa18a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:36.298350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26228b4-d735-4303-afdf-cc195b7aa18a None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:36.299040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26228b4-d735-4303-afdf-cc195b7aa18a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:36.300470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26228b4-d735-4303-afdf-cc195b7aa18a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:36.320667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 542/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:17:36 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:38.554473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee92141-d5f6-4bde-aa30-b355f9d83ac4 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:38.555033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee92141-d5f6-4bde-aa30-b355f9d83ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:38.555033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee92141-d5f6-4bde-aa30-b355f9d83ac4 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:38.556401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 569/1111] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:38.562249 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f2068-85bb-4533-a507-69030ab7b89d None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:38.562659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f2068-85bb-4533-a507-69030ab7b89d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:38.562805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f2068-85bb-4533-a507-69030ab7b89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:38.579827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-1a9f2068-85bb-4533-a507-69030ab7b89d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:38.581043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1a9f2068-85bb-4533-a507-69030ab7b89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:38.646452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 543/1112] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:38.674018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee92141-d5f6-4bde-aa30-b355f9d83ac4 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:38.700729 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:38.781723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee92141-d5f6-4bde-aa30-b355f9d83ac4 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:38.817584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d7a61-2f63-4fdf-9458-14c8292d98d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:38.818457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d7a61-2f63-4fdf-9458-14c8292d98d1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:38.818739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d7a61-2f63-4fdf-9458-14c8292d98d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:38.819014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d7a61-2f63-4fdf-9458-14c8292d98d1 None admin] PATH_INFO: `/v3/roles/e2a3f2f08d18410a954a4d6b8314562d` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:38.820529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d7a61-2f63-4fdf-9458-14c8292d98d1 None admin] RBAC: Authorizing `identity:get_role(role_id=e2a3f2f08d18410a954a4d6b8314562d)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:38.821970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d7a61-2f63-4fdf-9458-14c8292d98d1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:38.823548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 570/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:38 2022] GET /identity/v3/roles/e2a3f2f08d18410a954a4d6b8314562d => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:38.829733 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f2068-85bb-4533-a507-69030ab7b89d None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:38.895458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:38.926400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f2068-85bb-4533-a507-69030ab7b89d None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:38.958875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394b361-38e6-4e88-bc66-53977714eff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:38.959735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394b361-38e6-4e88-bc66-53977714eff5 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:38.960011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394b361-38e6-4e88-bc66-53977714eff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:38.960280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394b361-38e6-4e88-bc66-53977714eff5 None admin] PATH_INFO: `/v3/users/67bb1e06af9648289843af2589f8735a` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:38.972767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394b361-38e6-4e88-bc66-53977714eff5 None admin] RBAC: Authorizing `identity:get_user(user_id=67bb1e06af9648289843af2589f8735a)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:38.974241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394b361-38e6-4e88-bc66-53977714eff5 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:38.975859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 544/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:38 2022] GET /identity/v3/users/67bb1e06af9648289843af2589f8735a => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:38.981955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d7a61-2f63-4fdf-9458-14c8292d98d1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:39.069768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:39.078621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d7a61-2f63-4fdf-9458-14c8292d98d1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:39.111384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223df17d-1602-4908-87f6-d5bc8f0bf883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:39.112024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223df17d-1602-4908-87f6-d5bc8f0bf883 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:39.112309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223df17d-1602-4908-87f6-d5bc8f0bf883 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:39.112578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223df17d-1602-4908-87f6-d5bc8f0bf883 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:39.179113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223df17d-1602-4908-87f6-d5bc8f0bf883 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:39.180724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223df17d-1602-4908-87f6-d5bc8f0bf883 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:39.275529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:39.285318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 571/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:17:39.291395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394b361-38e6-4e88-bc66-53977714eff5 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:39.387830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394b361-38e6-4e88-bc66-53977714eff5 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:39.413217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:39.421432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac85e51-0dfd-488c-a223-d2fce25ff1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:39.422270 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac85e51-0dfd-488c-a223-d2fce25ff1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:39.422543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac85e51-0dfd-488c-a223-d2fce25ff1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:39.422808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac85e51-0dfd-488c-a223-d2fce25ff1d0 None admin] PATH_INFO: `/v3/projects/a407a0bd0f334e1aac397d76ada3a767` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:39.423958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac85e51-0dfd-488c-a223-d2fce25ff1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=a407a0bd0f334e1aac397d76ada3a767)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:39.425409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac85e51-0dfd-488c-a223-d2fce25ff1d0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:39.427058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 545/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:17:39 2022] GET /identity/v3/projects/a407a0bd0f334e1aac397d76ada3a767 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:39.433014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223df17d-1602-4908-87f6-d5bc8f0bf883 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:39.528492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223df17d-1602-4908-87f6-d5bc8f0bf883 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:39.560886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d292b-b8bc-44b6-81fc-9d5385b7ce5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:39.561569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d292b-b8bc-44b6-81fc-9d5385b7ce5e None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:39.561849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d292b-b8bc-44b6-81fc-9d5385b7ce5e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:39.562124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d292b-b8bc-44b6-81fc-9d5385b7ce5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:39.562956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d292b-b8bc-44b6-81fc-9d5385b7ce5e None admin] RBAC: Adding query filter params (role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=a407a0bd0f334e1aac397d76ada3a767, user.id=67bb1e06af9648289843af2589f8735a) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:39.563265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d292b-b8bc-44b6-81fc-9d5385b7ce5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:39.564697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d292b-b8bc-44b6-81fc-9d5385b7ce5e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:39.570724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 572/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:17:39 2022] GET /identity/v3/role_assignments?user.id=67bb1e06af9648289843af2589f8735a&scope.project.id=a407a0bd0f334e1aac397d76ada3a767&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:41.920715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5ed4d-8d6d-4cdf-85c8-7ceb93075aea None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:41.921253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5ed4d-8d6d-4cdf-85c8-7ceb93075aea None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:41.921253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5ed4d-8d6d-4cdf-85c8-7ceb93075aea None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:41.922586 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 546/1118] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:41.928065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ce58-0a94-4bf0-ac45-8ebbc0b9b2c7 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:41.928462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ce58-0a94-4bf0-ac45-8ebbc0b9b2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:41.928611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ce58-0a94-4bf0-ac45-8ebbc0b9b2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:41.945653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-3b88ce58-0a94-4bf0-ac45-8ebbc0b9b2c7 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:41.946834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3b88ce58-0a94-4bf0-ac45-8ebbc0b9b2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:41.994858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:42.012244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 573/1119] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:17:42.035031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5ed4d-8d6d-4cdf-85c8-7ceb93075aea None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:42.134567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5ed4d-8d6d-4cdf-85c8-7ceb93075aea None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:42.166714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2511fe4-71ce-41ce-b8cd-2e3ba9e3eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:42.168122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511fe4-71ce-41ce-b8cd-2e3ba9e3eba4 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:42.168401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511fe4-71ce-41ce-b8cd-2e3ba9e3eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:42.168667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511fe4-71ce-41ce-b8cd-2e3ba9e3eba4 None admin] PATH_INFO: `/v3/roles/e2a3f2f08d18410a954a4d6b8314562d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:42.170158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2511fe4-71ce-41ce-b8cd-2e3ba9e3eba4 None admin] RBAC: Authorizing `identity:get_role(role_id=e2a3f2f08d18410a954a4d6b8314562d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:42.171580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2511fe4-71ce-41ce-b8cd-2e3ba9e3eba4 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:42.172878 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 547/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:42 2022] GET /identity/v3/roles/e2a3f2f08d18410a954a4d6b8314562d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:42.178500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ce58-0a94-4bf0-ac45-8ebbc0b9b2c7 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:42.273550 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ce58-0a94-4bf0-ac45-8ebbc0b9b2c7 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:42.306569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad94c1-ce0b-4f04-bc66-3b5ef51f99a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:42.307424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad94c1-ce0b-4f04-bc66-3b5ef51f99a7 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:42.307705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad94c1-ce0b-4f04-bc66-3b5ef51f99a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:42.307994 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad94c1-ce0b-4f04-bc66-3b5ef51f99a7 None admin] PATH_INFO: `/v3/users/67bb1e06af9648289843af2589f8735a` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:42.310011 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad94c1-ce0b-4f04-bc66-3b5ef51f99a7 None admin] RBAC: Authorizing `identity:get_user(user_id=67bb1e06af9648289843af2589f8735a)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:42.311445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad94c1-ce0b-4f04-bc66-3b5ef51f99a7 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:42.313091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 574/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:42 2022] GET /identity/v3/users/67bb1e06af9648289843af2589f8735a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:42.318845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2511fe4-71ce-41ce-b8cd-2e3ba9e3eba4 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:42.380849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:42.422779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2511fe4-71ce-41ce-b8cd-2e3ba9e3eba4 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:42.456214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7749f5d-04cf-4cea-ae5a-0cb21aca5e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:42.457094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7749f5d-04cf-4cea-ae5a-0cb21aca5e12 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:42.457372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7749f5d-04cf-4cea-ae5a-0cb21aca5e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:42.457640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7749f5d-04cf-4cea-ae5a-0cb21aca5e12 None admin] PATH_INFO: `/v3/projects/a407a0bd0f334e1aac397d76ada3a767` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:42.458900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7749f5d-04cf-4cea-ae5a-0cb21aca5e12 None admin] RBAC: Authorizing `identity:get_project(project_id=a407a0bd0f334e1aac397d76ada3a767)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:42.460337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7749f5d-04cf-4cea-ae5a-0cb21aca5e12 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:42.462056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 548/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:17:42 2022] GET /identity/v3/projects/a407a0bd0f334e1aac397d76ada3a767 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:42.468607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad94c1-ce0b-4f04-bc66-3b5ef51f99a7 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:42.526059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:42.570675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad94c1-ce0b-4f04-bc66-3b5ef51f99a7 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:42.604034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5372876-6281-4ec7-aea7-66107f0fbe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:42.605633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5372876-6281-4ec7-aea7-66107f0fbe1a None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:42.605969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5372876-6281-4ec7-aea7-66107f0fbe1a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:42.607445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5372876-6281-4ec7-aea7-66107f0fbe1a None admin] PATH_INFO: `/v3/projects/a407a0bd0f334e1aac397d76ada3a767/users/67bb1e06af9648289843af2589f8735a/roles/e2a3f2f08d18410a954a4d6b8314562d` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:42.609808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5372876-6281-4ec7-aea7-66107f0fbe1a None admin] RBAC: Authorizing `identity:create_grant(project_id=a407a0bd0f334e1aac397d76ada3a767, user_id=67bb1e06af9648289843af2589f8735a, role_id=e2a3f2f08d18410a954a4d6b8314562d)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:42.611259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5372876-6281-4ec7-aea7-66107f0fbe1a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:42.623976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 575/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:17:42 2022] PUT /identity/v3/projects/a407a0bd0f334e1aac397d76ada3a767/users/67bb1e06af9648289843af2589f8735a/roles/e2a3f2f08d18410a954a4d6b8314562d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:17:44.937789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf6dfa-525b-4948-b08e-21043e223371 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:44.938300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf6dfa-525b-4948-b08e-21043e223371 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:44.938300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf6dfa-525b-4948-b08e-21043e223371 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:44.939644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 549/1124] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:44.944704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9283540-fd54-49c1-af6a-f7ea6c4acf2d None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:44.945092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9283540-fd54-49c1-af6a-f7ea6c4acf2d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:44.945287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9283540-fd54-49c1-af6a-f7ea6c4acf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:44.962267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a9283540-fd54-49c1-af6a-f7ea6c4acf2d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:44.963458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a9283540-fd54-49c1-af6a-f7ea6c4acf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:45.028384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 576/1125] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:45.051426 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf6dfa-525b-4948-b08e-21043e223371 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:45.149972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf6dfa-525b-4948-b08e-21043e223371 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:45.178289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:45.183508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08220ab6-f3ed-420d-913c-7cb4ae6275bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:45.184385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08220ab6-f3ed-420d-913c-7cb4ae6275bf None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:45.184662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08220ab6-f3ed-420d-913c-7cb4ae6275bf None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:45.184956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08220ab6-f3ed-420d-913c-7cb4ae6275bf None admin] PATH_INFO: `/v3/roles/e2a3f2f08d18410a954a4d6b8314562d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:45.186416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08220ab6-f3ed-420d-913c-7cb4ae6275bf None admin] RBAC: Authorizing `identity:get_role(role_id=e2a3f2f08d18410a954a4d6b8314562d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:45.187869 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08220ab6-f3ed-420d-913c-7cb4ae6275bf None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:45.189201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 550/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:45 2022] GET /identity/v3/roles/e2a3f2f08d18410a954a4d6b8314562d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:45.194870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9283540-fd54-49c1-af6a-f7ea6c4acf2d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:45.238461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:45.296492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9283540-fd54-49c1-af6a-f7ea6c4acf2d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:45.330543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838523a-3be1-4e21-ae3c-47030c533994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:45.331408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838523a-3be1-4e21-ae3c-47030c533994 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:45.331691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838523a-3be1-4e21-ae3c-47030c533994 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:45.331968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838523a-3be1-4e21-ae3c-47030c533994 None admin] PATH_INFO: `/v3/users/67bb1e06af9648289843af2589f8735a` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:45.333170 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838523a-3be1-4e21-ae3c-47030c533994 None admin] RBAC: Authorizing `identity:get_user(user_id=67bb1e06af9648289843af2589f8735a)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:45.334593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838523a-3be1-4e21-ae3c-47030c533994 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:45.336225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 577/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:45 2022] GET /identity/v3/users/67bb1e06af9648289843af2589f8735a => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:45.342248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08220ab6-f3ed-420d-913c-7cb4ae6275bf None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:45.439340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08220ab6-f3ed-420d-913c-7cb4ae6275bf None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:45.471484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4511d92-d137-4dd7-bf90-5474f0592c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:45.472114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4511d92-d137-4dd7-bf90-5474f0592c9a None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:45.472390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4511d92-d137-4dd7-bf90-5474f0592c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:45.472657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4511d92-d137-4dd7-bf90-5474f0592c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:45.534592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:45.536902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4511d92-d137-4dd7-bf90-5474f0592c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:45.538650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4511d92-d137-4dd7-bf90-5474f0592c9a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:45.633400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 551/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:17:45.640027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838523a-3be1-4e21-ae3c-47030c533994 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:45.736462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838523a-3be1-4e21-ae3c-47030c533994 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:45.749716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:45.771464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f7d95-3ac6-4b4b-8a54-17a6c038c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:45.772328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f7d95-3ac6-4b4b-8a54-17a6c038c3ff None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:45.772601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f7d95-3ac6-4b4b-8a54-17a6c038c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:45.772885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f7d95-3ac6-4b4b-8a54-17a6c038c3ff None admin] PATH_INFO: `/v3/projects/a407a0bd0f334e1aac397d76ada3a767` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:45.774059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f7d95-3ac6-4b4b-8a54-17a6c038c3ff None admin] RBAC: Authorizing `identity:get_project(project_id=a407a0bd0f334e1aac397d76ada3a767)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:45.775474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f7d95-3ac6-4b4b-8a54-17a6c038c3ff None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:45.777240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 578/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:17:45 2022] GET /identity/v3/projects/a407a0bd0f334e1aac397d76ada3a767 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:45.783488 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4511d92-d137-4dd7-bf90-5474f0592c9a None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:45.877291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:45.880655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4511d92-d137-4dd7-bf90-5474f0592c9a None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:45.913628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1a108-8b5a-44a8-999b-af693ee189c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:45.914271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1a108-8b5a-44a8-999b-af693ee189c1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:45.914556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1a108-8b5a-44a8-999b-af693ee189c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:45.914833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1a108-8b5a-44a8-999b-af693ee189c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:45.915655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1a108-8b5a-44a8-999b-af693ee189c1 None admin] RBAC: Adding query filter params (role.id=e2a3f2f08d18410a954a4d6b8314562d, scope.project.id=a407a0bd0f334e1aac397d76ada3a767, user.id=67bb1e06af9648289843af2589f8735a) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:45.915966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1a108-8b5a-44a8-999b-af693ee189c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:45.917437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1a108-8b5a-44a8-999b-af693ee189c1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:45.923647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 552/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:17:45 2022] GET /identity/v3/role_assignments?user.id=67bb1e06af9648289843af2589f8735a&scope.project.id=a407a0bd0f334e1aac397d76ada3a767&role.id=e2a3f2f08d18410a954a4d6b8314562d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:48.105069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896e323-03c7-462c-b7c4-769acead2f5e None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:48.105617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896e323-03c7-462c-b7c4-769acead2f5e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:48.105750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896e323-03c7-462c-b7c4-769acead2f5e None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:48.107127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 579/1131] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:48.112257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e29546-83a9-48b1-825f-54fa6ff9f39e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:48.112662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e29546-83a9-48b1-825f-54fa6ff9f39e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:48.112817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e29546-83a9-48b1-825f-54fa6ff9f39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:48.130125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-c9e29546-83a9-48b1-825f-54fa6ff9f39e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:48.131309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c9e29546-83a9-48b1-825f-54fa6ff9f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:48.197561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 553/1132] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:48.221759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896e323-03c7-462c-b7c4-769acead2f5e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:48.319826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896e323-03c7-462c-b7c4-769acead2f5e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:48.353195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf62b4-64e4-49f0-b726-661965d115dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:48.353977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf62b4-64e4-49f0-b726-661965d115dc None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:48.354260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf62b4-64e4-49f0-b726-661965d115dc None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:48.354536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf62b4-64e4-49f0-b726-661965d115dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:48.355780 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf62b4-64e4-49f0-b726-661965d115dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:48.357274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf62b4-64e4-49f0-b726-661965d115dc None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:48.358906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 580/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:48.364581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e29546-83a9-48b1-825f-54fa6ff9f39e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:48.454666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:48.461303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e29546-83a9-48b1-825f-54fa6ff9f39e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:48.493609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc39898-56ee-45e0-be23-8ed2039b0ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:48.494276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc39898-56ee-45e0-be23-8ed2039b0ee2 None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:48.494559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc39898-56ee-45e0-be23-8ed2039b0ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:48.494835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc39898-56ee-45e0-be23-8ed2039b0ee2 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:48.495530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc39898-56ee-45e0-be23-8ed2039b0ee2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:48.497728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc39898-56ee-45e0-be23-8ed2039b0ee2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:48.523555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 554/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:17:48 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:50.780051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a278cb-c5b9-430d-b0d9-33be1aeb3723 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:50.780598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a278cb-c5b9-430d-b0d9-33be1aeb3723 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:50.780598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a278cb-c5b9-430d-b0d9-33be1aeb3723 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:50.782030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 581/1135] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:50.787424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668a561-f106-4dfa-913b-b8ec664e162f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:50.787815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668a561-f106-4dfa-913b-b8ec664e162f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:50.787990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668a561-f106-4dfa-913b-b8ec664e162f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:50.804383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a668a561-f106-4dfa-913b-b8ec664e162f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:50.805610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a668a561-f106-4dfa-913b-b8ec664e162f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:50.871670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 555/1136] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:50.896762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a278cb-c5b9-430d-b0d9-33be1aeb3723 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:50.946303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:50.996772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a278cb-c5b9-430d-b0d9-33be1aeb3723 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:51.029514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851ab82-1074-4370-a33b-aa56918fec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:51.030295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851ab82-1074-4370-a33b-aa56918fec09 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:51.030578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851ab82-1074-4370-a33b-aa56918fec09 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:51.030852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851ab82-1074-4370-a33b-aa56918fec09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:51.032104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851ab82-1074-4370-a33b-aa56918fec09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:51.033580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851ab82-1074-4370-a33b-aa56918fec09 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:51.035220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 582/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:17:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:51.040776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668a561-f106-4dfa-913b-b8ec664e162f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:51.115414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:51.139124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668a561-f106-4dfa-913b-b8ec664e162f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:51.171774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade65092-194b-4f70-9a3a-961b6e771a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:51.172430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade65092-194b-4f70-9a3a-961b6e771a6a None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:51.172712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade65092-194b-4f70-9a3a-961b6e771a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:51.173023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade65092-194b-4f70-9a3a-961b6e771a6a None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:51.173735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade65092-194b-4f70-9a3a-961b6e771a6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:51.175191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade65092-194b-4f70-9a3a-961b6e771a6a None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:51.194645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 556/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:17:51 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:17:53.424839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a8a7a-4dd4-4739-ba02-35e968961460 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:53.425420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a8a7a-4dd4-4739-ba02-35e968961460 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:53.425420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a8a7a-4dd4-4739-ba02-35e968961460 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:53.426796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 583/1139] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:53.431930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b203fe0-21b7-4f10-a467-2162eb66c120 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:53.432310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b203fe0-21b7-4f10-a467-2162eb66c120 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:53.432482 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b203fe0-21b7-4f10-a467-2162eb66c120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:53.449630 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-1b203fe0-21b7-4f10-a467-2162eb66c120 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:53.450806 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1b203fe0-21b7-4f10-a467-2162eb66c120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:53.515372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 557/1140] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:53.538259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a8a7a-4dd4-4739-ba02-35e968961460 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:53.646902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a8a7a-4dd4-4739-ba02-35e968961460 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:53.665261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:53.683743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9fa8c-f184-430d-a4fc-e28b3e450152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:53.684479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9fa8c-f184-430d-a4fc-e28b3e450152 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:53.684760 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9fa8c-f184-430d-a4fc-e28b3e450152 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:53.685069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9fa8c-f184-430d-a4fc-e28b3e450152 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:53.694899 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9fa8c-f184-430d-a4fc-e28b3e450152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-8cf9fa8c-f184-430d-a4fc-e28b3e450152 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:53.702534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:53.703583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:53.704668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 584/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:17:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:53.709767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b203fe0-21b7-4f10-a467-2162eb66c120 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:53.769017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:53.808139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b203fe0-21b7-4f10-a467-2162eb66c120 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:53.840538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb7b3c-cf06-42d1-88ce-6f3335fb1f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:53.841216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb7b3c-cf06-42d1-88ce-6f3335fb1f66 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:53.841493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb7b3c-cf06-42d1-88ce-6f3335fb1f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:53.841765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb7b3c-cf06-42d1-88ce-6f3335fb1f66 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:53.842505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb7b3c-cf06-42d1-88ce-6f3335fb1f66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:53.842814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb7b3c-cf06-42d1-88ce-6f3335fb1f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:53.844174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb7b3c-cf06-42d1-88ce-6f3335fb1f66 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:53.852061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 558/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:17:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:53.858463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9fa8c-f184-430d-a4fc-e28b3e450152 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:53.954544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9fa8c-f184-430d-a4fc-e28b3e450152 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:53.986883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77b586-f983-4a97-98c5-947430fc3943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:53.987735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b586-f983-4a97-98c5-947430fc3943 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:53.988017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b586-f983-4a97-98c5-947430fc3943 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:53.988291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b586-f983-4a97-98c5-947430fc3943 None admin] PATH_INFO: `/v3/users/1f1636bc7f074b19aa21c9571a3bf40b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:54.000914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77b586-f983-4a97-98c5-947430fc3943 None admin] RBAC: Authorizing `identity:get_user(user_id=1f1636bc7f074b19aa21c9571a3bf40b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:54.002357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77b586-f983-4a97-98c5-947430fc3943 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:54.004240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 585/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:53 2022] GET /identity/v3/users/1f1636bc7f074b19aa21c9571a3bf40b => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:54.009834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb7b3c-cf06-42d1-88ce-6f3335fb1f66 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:54.108924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb7b3c-cf06-42d1-88ce-6f3335fb1f66 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:54.131088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:54.142136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c122873-65d7-4bef-bc9e-f629da9c24c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:54.142780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c122873-65d7-4bef-bc9e-f629da9c24c9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:54.143059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c122873-65d7-4bef-bc9e-f629da9c24c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:54.143338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c122873-65d7-4bef-bc9e-f629da9c24c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:54.207947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c122873-65d7-4bef-bc9e-f629da9c24c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:54.209372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c122873-65d7-4bef-bc9e-f629da9c24c9 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:54.304159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 559/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:17:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:17:54.310671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77b586-f983-4a97-98c5-947430fc3943 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:54.344059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:54.408832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77b586-f983-4a97-98c5-947430fc3943 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:54.441271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfd39d-1ec1-44fe-b31b-788953550a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:54.442121 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfd39d-1ec1-44fe-b31b-788953550a7c None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:54.442401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfd39d-1ec1-44fe-b31b-788953550a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:54.442673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfd39d-1ec1-44fe-b31b-788953550a7c None admin] PATH_INFO: `/v3/projects/eed6d3ecb6c44e57ad4da2d9544defe1` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:54.443829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfd39d-1ec1-44fe-b31b-788953550a7c None admin] RBAC: Authorizing `identity:get_project(project_id=eed6d3ecb6c44e57ad4da2d9544defe1)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:54.445276 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfd39d-1ec1-44fe-b31b-788953550a7c None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:54.446943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 586/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:17:54 2022] GET /identity/v3/projects/eed6d3ecb6c44e57ad4da2d9544defe1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:54.453557 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c122873-65d7-4bef-bc9e-f629da9c24c9 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:54.476971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:54.548912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c122873-65d7-4bef-bc9e-f629da9c24c9 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:54.581822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f684ba1-eb72-4d32-8a76-5e72f5c3c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:54.582457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f684ba1-eb72-4d32-8a76-5e72f5c3c879 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:54.582735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f684ba1-eb72-4d32-8a76-5e72f5c3c879 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:54.583007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f684ba1-eb72-4d32-8a76-5e72f5c3c879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:54.583821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f684ba1-eb72-4d32-8a76-5e72f5c3c879 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=eed6d3ecb6c44e57ad4da2d9544defe1, user.id=1f1636bc7f074b19aa21c9571a3bf40b) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:54.584129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f684ba1-eb72-4d32-8a76-5e72f5c3c879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:54.585590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f684ba1-eb72-4d32-8a76-5e72f5c3c879 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:54.591076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 560/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:17:54 2022] GET /identity/v3/role_assignments?user.id=1f1636bc7f074b19aa21c9571a3bf40b&scope.project.id=eed6d3ecb6c44e57ad4da2d9544defe1&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:17:56.787154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4c882-7d30-4604-a3c9-faed57cb8357 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:56.787688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4c882-7d30-4604-a3c9-faed57cb8357 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:56.787688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4c882-7d30-4604-a3c9-faed57cb8357 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:56.789107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 587/1147] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:56.794417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195d960-9a5d-4b51-a254-b930f9817cc1 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:56.794856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195d960-9a5d-4b51-a254-b930f9817cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:56.794962 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195d960-9a5d-4b51-a254-b930f9817cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:56.812180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8195d960-9a5d-4b51-a254-b930f9817cc1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:56.813371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8195d960-9a5d-4b51-a254-b930f9817cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:56.878587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 561/1148] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:17:56.901308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4c882-7d30-4604-a3c9-faed57cb8357 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:57.000973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4c882-7d30-4604-a3c9-faed57cb8357 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:57.003173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:57.042274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb015ea0-128f-47e0-be30-05e89d12d06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:57.043064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb015ea0-128f-47e0-be30-05e89d12d06d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:57.043349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb015ea0-128f-47e0-be30-05e89d12d06d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:57.043626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb015ea0-128f-47e0-be30-05e89d12d06d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:57.053533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb015ea0-128f-47e0-be30-05e89d12d06d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-fb015ea0-128f-47e0-be30-05e89d12d06d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:17:57.061229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:17:57.062383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:17:57.063574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 588/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:17:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:17:57.068442 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195d960-9a5d-4b51-a254-b930f9817cc1 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:57.086709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:57.170395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195d960-9a5d-4b51-a254-b930f9817cc1 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:57.205473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f21143-fa8d-4443-9371-7eb547995878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:57.206187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f21143-fa8d-4443-9371-7eb547995878 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:57.206504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f21143-fa8d-4443-9371-7eb547995878 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:57.206812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f21143-fa8d-4443-9371-7eb547995878 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:57.207555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f21143-fa8d-4443-9371-7eb547995878 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:17:57.207892 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f21143-fa8d-4443-9371-7eb547995878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:57.209422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f21143-fa8d-4443-9371-7eb547995878 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:57.218136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 562/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:17:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:57.223938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb015ea0-128f-47e0-be30-05e89d12d06d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:57.322221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb015ea0-128f-47e0-be30-05e89d12d06d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:57.354492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c106db-b0d6-40f0-bfd1-8f3121595a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:57.355354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c106db-b0d6-40f0-bfd1-8f3121595a27 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:57.355638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c106db-b0d6-40f0-bfd1-8f3121595a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:57.355916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c106db-b0d6-40f0-bfd1-8f3121595a27 None admin] PATH_INFO: `/v3/users/1f1636bc7f074b19aa21c9571a3bf40b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:57.357140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c106db-b0d6-40f0-bfd1-8f3121595a27 None admin] RBAC: Authorizing `identity:get_user(user_id=1f1636bc7f074b19aa21c9571a3bf40b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:57.358564 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c106db-b0d6-40f0-bfd1-8f3121595a27 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:57.360178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 589/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:17:57 2022] GET /identity/v3/users/1f1636bc7f074b19aa21c9571a3bf40b => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:57.366067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f21143-fa8d-4443-9371-7eb547995878 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:57.451370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:57.470151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f21143-fa8d-4443-9371-7eb547995878 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:57.502745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4084f-44d0-4640-b549-58ed7a29884c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:57.503840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4084f-44d0-4640-b549-58ed7a29884c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:57.504122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4084f-44d0-4640-b549-58ed7a29884c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:57.504395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4084f-44d0-4640-b549-58ed7a29884c None admin] PATH_INFO: `/v3/projects/eed6d3ecb6c44e57ad4da2d9544defe1` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:57.505545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4084f-44d0-4640-b549-58ed7a29884c None admin] RBAC: Authorizing `identity:get_project(project_id=eed6d3ecb6c44e57ad4da2d9544defe1)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:57.506973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4084f-44d0-4640-b549-58ed7a29884c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:57.508633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 563/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:17:57 2022] GET /identity/v3/projects/eed6d3ecb6c44e57ad4da2d9544defe1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:17:57.515000 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c106db-b0d6-40f0-bfd1-8f3121595a27 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:17:57.544494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:17:57.612705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c106db-b0d6-40f0-bfd1-8f3121595a27 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:17:57.646193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708d9f9-5fcd-44da-98a8-2b329335a1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:17:57.647081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708d9f9-5fcd-44da-98a8-2b329335a1a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:57.647363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708d9f9-5fcd-44da-98a8-2b329335a1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:57.647636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708d9f9-5fcd-44da-98a8-2b329335a1a2 None admin] PATH_INFO: `/v3/projects/eed6d3ecb6c44e57ad4da2d9544defe1/users/1f1636bc7f074b19aa21c9571a3bf40b/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:57.649575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708d9f9-5fcd-44da-98a8-2b329335a1a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=eed6d3ecb6c44e57ad4da2d9544defe1, user_id=1f1636bc7f074b19aa21c9571a3bf40b, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:17:57.650998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708d9f9-5fcd-44da-98a8-2b329335a1a2 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:17:57.663655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 590/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:17:57 2022] PUT /identity/v3/projects/eed6d3ecb6c44e57ad4da2d9544defe1/users/1f1636bc7f074b19aa21c9571a3bf40b/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:17:59.898246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044d80-34f2-47a1-b7c1-fc934669fe9b None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:59.898793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044d80-34f2-47a1-b7c1-fc934669fe9b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:59.898793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044d80-34f2-47a1-b7c1-fc934669fe9b None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:59.900099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 564/1154] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:17:59.905776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57887db-ba18-4d77-b431-da6004ffe025 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:17:59.906211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57887db-ba18-4d77-b431-da6004ffe025 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:17:59.906317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57887db-ba18-4d77-b431-da6004ffe025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:17:59.923369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d57887db-ba18-4d77-b431-da6004ffe025 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:17:59.924540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d57887db-ba18-4d77-b431-da6004ffe025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:17:59.989797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 591/1155] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:17:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:18:00.013609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044d80-34f2-47a1-b7c1-fc934669fe9b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:00.112434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044d80-34f2-47a1-b7c1-fc934669fe9b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:00.145275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b522c8-2a9b-4e40-981c-c857bcb55244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:00.147359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b522c8-2a9b-4e40-981c-c857bcb55244 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:00.147643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b522c8-2a9b-4e40-981c-c857bcb55244 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:00.147919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b522c8-2a9b-4e40-981c-c857bcb55244 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:00.158438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b522c8-2a9b-4e40-981c-c857bcb55244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-62b522c8-2a9b-4e40-981c-c857bcb55244 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:18:00.165846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:18:00.167064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:18:00.168331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 565/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:18:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:18:00.173715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57887db-ba18-4d77-b431-da6004ffe025 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:00.194844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:00.271523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57887db-ba18-4d77-b431-da6004ffe025 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:00.304184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f425-f726-4971-b183-156688d011dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:00.304844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f425-f726-4971-b183-156688d011dc None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:00.305221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f425-f726-4971-b183-156688d011dc None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:00.305499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f425-f726-4971-b183-156688d011dc None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:00.306238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6f425-f726-4971-b183-156688d011dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:18:00.306543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6f425-f726-4971-b183-156688d011dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:00.308620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6f425-f726-4971-b183-156688d011dc None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:00.316460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 592/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:18:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:00.322274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b522c8-2a9b-4e40-981c-c857bcb55244 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:00.334098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:00.421491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b522c8-2a9b-4e40-981c-c857bcb55244 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:00.453888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6aa82-b894-44b9-a087-cab2581efbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:00.454730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6aa82-b894-44b9-a087-cab2581efbad None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:00.455008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6aa82-b894-44b9-a087-cab2581efbad None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:00.455277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6aa82-b894-44b9-a087-cab2581efbad None admin] PATH_INFO: `/v3/users/1f1636bc7f074b19aa21c9571a3bf40b` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:00.456425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6aa82-b894-44b9-a087-cab2581efbad None admin] RBAC: Authorizing `identity:get_user(user_id=1f1636bc7f074b19aa21c9571a3bf40b)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:00.457894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6aa82-b894-44b9-a087-cab2581efbad None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:00.459495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 566/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:18:00 2022] GET /identity/v3/users/1f1636bc7f074b19aa21c9571a3bf40b => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:00.465354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f425-f726-4971-b183-156688d011dc None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:00.544737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:00.563071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f425-f726-4971-b183-156688d011dc None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:00.595783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aad3c-d29f-40e2-a91f-2e62291070dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:00.596432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aad3c-d29f-40e2-a91f-2e62291070dc None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:00.596717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aad3c-d29f-40e2-a91f-2e62291070dc None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:00.597030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aad3c-d29f-40e2-a91f-2e62291070dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:00.662256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8aad3c-d29f-40e2-a91f-2e62291070dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:00.663698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8aad3c-d29f-40e2-a91f-2e62291070dc None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:00.742987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:00.762218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 593/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:18:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:18:00.768244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6aa82-b894-44b9-a087-cab2581efbad None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:00.843588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:00.866473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6aa82-b894-44b9-a087-cab2581efbad None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:00.899766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05453e3a-b7a6-4c94-b42d-e630338e6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:00.900618 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05453e3a-b7a6-4c94-b42d-e630338e6b52 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:00.901133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05453e3a-b7a6-4c94-b42d-e630338e6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:00.901424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05453e3a-b7a6-4c94-b42d-e630338e6b52 None admin] PATH_INFO: `/v3/projects/eed6d3ecb6c44e57ad4da2d9544defe1` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:00.902599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05453e3a-b7a6-4c94-b42d-e630338e6b52 None admin] RBAC: Authorizing `identity:get_project(project_id=eed6d3ecb6c44e57ad4da2d9544defe1)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:00.904031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05453e3a-b7a6-4c94-b42d-e630338e6b52 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:00.905743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 567/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:18:00 2022] GET /identity/v3/projects/eed6d3ecb6c44e57ad4da2d9544defe1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:00.911759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aad3c-d29f-40e2-a91f-2e62291070dc None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:01.008463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aad3c-d29f-40e2-a91f-2e62291070dc None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:01.040557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb9390-4765-4239-a895-9b6650feb617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:01.041229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb9390-4765-4239-a895-9b6650feb617 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:01.042049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb9390-4765-4239-a895-9b6650feb617 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:01.042477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb9390-4765-4239-a895-9b6650feb617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:01.043298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb9390-4765-4239-a895-9b6650feb617 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=eed6d3ecb6c44e57ad4da2d9544defe1, user.id=1f1636bc7f074b19aa21c9571a3bf40b) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:18:01.043606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb9390-4765-4239-a895-9b6650feb617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:01.045055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb9390-4765-4239-a895-9b6650feb617 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:01.050842 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 594/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:18:00 2022] GET /identity/v3/role_assignments?user.id=1f1636bc7f074b19aa21c9571a3bf40b&scope.project.id=eed6d3ecb6c44e57ad4da2d9544defe1&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:03.265944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1360afc-e8fe-459b-b7db-9b9efc5ef88f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:03.266502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1360afc-e8fe-459b-b7db-9b9efc5ef88f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:03.266502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1360afc-e8fe-459b-b7db-9b9efc5ef88f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:03.267875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 568/1162] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:18:03.273179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13550b2e-e8c5-466b-9651-8635addee741 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:03.273588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13550b2e-e8c5-466b-9651-8635addee741 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:03.273743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13550b2e-e8c5-466b-9651-8635addee741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:03.291583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-13550b2e-e8c5-466b-9651-8635addee741 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:18:03.292790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-13550b2e-e8c5-466b-9651-8635addee741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:18:03.333832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:03.358050 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 595/1163] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:18:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:18:03.380828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1360afc-e8fe-459b-b7db-9b9efc5ef88f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:03.483328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1360afc-e8fe-459b-b7db-9b9efc5ef88f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:03.519074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017b8aa-1531-40bb-b92b-eb15b6abe926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:03.519726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017b8aa-1531-40bb-b92b-eb15b6abe926 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:03.520003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017b8aa-1531-40bb-b92b-eb15b6abe926 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:03.520273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017b8aa-1531-40bb-b92b-eb15b6abe926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:03.534278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:03.588435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017b8aa-1531-40bb-b92b-eb15b6abe926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:03.590157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017b8aa-1531-40bb-b92b-eb15b6abe926 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:03.694704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 569/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:18:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:18:03.695915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13550b2e-e8c5-466b-9651-8635addee741 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:03.793387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13550b2e-e8c5-466b-9651-8635addee741 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:03.826766 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41648b3-de19-47de-986b-19c30d326bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:03.827557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41648b3-de19-47de-986b-19c30d326bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:03.827840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41648b3-de19-47de-986b-19c30d326bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:03.828117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41648b3-de19-47de-986b-19c30d326bc4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:03.834705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41648b3-de19-47de-986b-19c30d326bc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:03.836130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41648b3-de19-47de-986b-19c30d326bc4 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-c41648b3-de19-47de-986b-19c30d326bc4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:18:03.850423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:18:03.856894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:18:03.857997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 596/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:18:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:18:03.858987 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017b8aa-1531-40bb-b92b-eb15b6abe926 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:03.902657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:03.956452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017b8aa-1531-40bb-b92b-eb15b6abe926 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:03.989621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cabb9e-6e4b-4c82-a112-7ebbe448c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:03.990273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cabb9e-6e4b-4c82-a112-7ebbe448c1de None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:03.990563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cabb9e-6e4b-4c82-a112-7ebbe448c1de None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:03.990841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cabb9e-6e4b-4c82-a112-7ebbe448c1de None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:03.991561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cabb9e-6e4b-4c82-a112-7ebbe448c1de None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:18:03.991870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cabb9e-6e4b-4c82-a112-7ebbe448c1de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:03.993275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cabb9e-6e4b-4c82-a112-7ebbe448c1de None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:04.000942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 570/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Mon May 23 15:18:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:18:06.190617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062456fe-9da5-490c-8adb-7456aac4bf22 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:06.191176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062456fe-9da5-490c-8adb-7456aac4bf22 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:06.191176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062456fe-9da5-490c-8adb-7456aac4bf22 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:06.192564 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 597/1167] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:18:06.198688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c807ad5-49a8-42de-a106-e833e8fe8251 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:06.199094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c807ad5-49a8-42de-a106-e833e8fe8251 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:06.199230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c807ad5-49a8-42de-a106-e833e8fe8251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:06.215981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4c807ad5-49a8-42de-a106-e833e8fe8251 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:18:06.217190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4c807ad5-49a8-42de-a106-e833e8fe8251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:18:06.281071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 571/1168] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:18:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:18:06.303164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062456fe-9da5-490c-8adb-7456aac4bf22 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:06.339104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:06.406356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062456fe-9da5-490c-8adb-7456aac4bf22 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:06.438537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c120f-b934-44ec-911d-54f79920803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:06.439187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c120f-b934-44ec-911d-54f79920803e None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:06.439469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c120f-b934-44ec-911d-54f79920803e None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:06.439742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c120f-b934-44ec-911d-54f79920803e None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:06.440468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c120f-b934-44ec-911d-54f79920803e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:06.441914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c120f-b934-44ec-911d-54f79920803e None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:06.455777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 598/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Mon May 23 15:18:06 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:18:08.640040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9a5a7-4eff-4603-8aef-0be20af63704 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:08.640496 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9a5a7-4eff-4603-8aef-0be20af63704 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:08.640606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9a5a7-4eff-4603-8aef-0be20af63704 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:08.641997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 572/1170] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:18:08.647411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a4ff-9c78-418d-b266-4c2284c4f161 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:08.647827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a4ff-9c78-418d-b266-4c2284c4f161 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:08.647978 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a4ff-9c78-418d-b266-4c2284c4f161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:08.664384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6195a4ff-9c78-418d-b266-4c2284c4f161 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:18:08.665611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6195a4ff-9c78-418d-b266-4c2284c4f161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:18:08.731516 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 599/1171] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:18:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:18:08.757896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9a5a7-4eff-4603-8aef-0be20af63704 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:08.801558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:08.860038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9a5a7-4eff-4603-8aef-0be20af63704 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:08.893001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033009ec-5f6b-4d9e-8ca9-7c813fbc872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:08.893840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033009ec-5f6b-4d9e-8ca9-7c813fbc872b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:08.894114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033009ec-5f6b-4d9e-8ca9-7c813fbc872b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:08.894386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033009ec-5f6b-4d9e-8ca9-7c813fbc872b None admin] PATH_INFO: `/v3/domains/de7a1a92a30e480d84869961ba5eb80c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:08.904466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033009ec-5f6b-4d9e-8ca9-7c813fbc872b None admin] RBAC: Authorizing `identity:get_domain(domain_id=de7a1a92a30e480d84869961ba5eb80c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:08.905941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033009ec-5f6b-4d9e-8ca9-7c813fbc872b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:08.907546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 573/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Mon May 23 15:18:08 2022] GET /identity/v3/domains/de7a1a92a30e480d84869961ba5eb80c => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:08.913393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195a4ff-9c78-418d-b266-4c2284c4f161 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:08.939950 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:09.013224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195a4ff-9c78-418d-b266-4c2284c4f161 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:09.045879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdd408-db05-4d36-8f84-79b0602e1c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:09.046527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdd408-db05-4d36-8f84-79b0602e1c15 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:09.046807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdd408-db05-4d36-8f84-79b0602e1c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:09.047081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdd408-db05-4d36-8f84-79b0602e1c15 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:09.047784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdd408-db05-4d36-8f84-79b0602e1c15 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:09.049226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdd408-db05-4d36-8f84-79b0602e1c15 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:09.074012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 600/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:18:08 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:18:11.336435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fcfc3-324a-4371-a173-89c6849eb53c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:11.337021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fcfc3-324a-4371-a173-89c6849eb53c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:11.337129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fcfc3-324a-4371-a173-89c6849eb53c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:11.339537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 574/1174] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:18:11.344952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a49161-0dae-43fb-9e77-4c2b50add609 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:11.345362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a49161-0dae-43fb-9e77-4c2b50add609 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:11.345510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a49161-0dae-43fb-9e77-4c2b50add609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:11.362084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-e7a49161-0dae-43fb-9e77-4c2b50add609 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:18:11.363252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e7a49161-0dae-43fb-9e77-4c2b50add609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:18:11.427423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 601/1175] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:18:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:18:11.452269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fcfc3-324a-4371-a173-89c6849eb53c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:11.547164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:11.551958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fcfc3-324a-4371-a173-89c6849eb53c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:11.585224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff1913-f04b-4451-8299-823f25f99d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:11.586069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff1913-f04b-4451-8299-823f25f99d5c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:11.586344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff1913-f04b-4451-8299-823f25f99d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:11.586807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff1913-f04b-4451-8299-823f25f99d5c None admin] PATH_INFO: `/v3/domains/de7a1a92a30e480d84869961ba5eb80c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:11.588078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff1913-f04b-4451-8299-823f25f99d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=de7a1a92a30e480d84869961ba5eb80c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:11.589547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff1913-f04b-4451-8299-823f25f99d5c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:11.591154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 575/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Mon May 23 15:18:11 2022] GET /identity/v3/domains/de7a1a92a30e480d84869961ba5eb80c => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:11.596761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a49161-0dae-43fb-9e77-4c2b50add609 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:11.606908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:11.693271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a49161-0dae-43fb-9e77-4c2b50add609 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:11.725420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b834733-5a12-43f0-9253-06d6968471c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:11.726070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b834733-5a12-43f0-9253-06d6968471c6 None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:11.726347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b834733-5a12-43f0-9253-06d6968471c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:11.726616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b834733-5a12-43f0-9253-06d6968471c6 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:11.727975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b834733-5a12-43f0-9253-06d6968471c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:11.729461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b834733-5a12-43f0-9253-06d6968471c6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:11.749468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 602/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:18:11 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:18:14.002963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d4864-89be-4c53-a76e-d73a0bceb871 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:14.003461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d4864-89be-4c53-a76e-d73a0bceb871 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:14.003461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d4864-89be-4c53-a76e-d73a0bceb871 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:14.004803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 576/1178] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:18:14.010086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d199-66d2-45a4-974a-239065f85137 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:14.010519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d199-66d2-45a4-974a-239065f85137 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:14.010652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d199-66d2-45a4-974a-239065f85137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:14.027650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-c825d199-66d2-45a4-974a-239065f85137 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:18:14.028836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-c825d199-66d2-45a4-974a-239065f85137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:18:14.085650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:14.096260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 603/1179] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:18:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:18:14.120409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d4864-89be-4c53-a76e-d73a0bceb871 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:14.218778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d4864-89be-4c53-a76e-d73a0bceb871 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:14.250489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d179ad7-b9d9-45db-981d-0fd6adbd5fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:14.251512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d179ad7-b9d9-45db-981d-0fd6adbd5fdd None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:14.251787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d179ad7-b9d9-45db-981d-0fd6adbd5fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:14.252055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d179ad7-b9d9-45db-981d-0fd6adbd5fdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:14.262806 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d179ad7-b9d9-45db-981d-0fd6adbd5fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-9d179ad7-b9d9-45db-981d-0fd6adbd5fdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:18:14.270280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:18:14.271453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:18:14.272489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 577/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:18:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:18:14.278121 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825d199-66d2-45a4-974a-239065f85137 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:14.374071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825d199-66d2-45a4-974a-239065f85137 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:14.407696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6b8e-3576-4d86-a791-1e971fd693a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:14.408349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6b8e-3576-4d86-a791-1e971fd693a6 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:14.408630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6b8e-3576-4d86-a791-1e971fd693a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:14.408928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6b8e-3576-4d86-a791-1e971fd693a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:14.409673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6b8e-3576-4d86-a791-1e971fd693a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:18:14.409980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6b8e-3576-4d86-a791-1e971fd693a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:14.411329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6b8e-3576-4d86-a791-1e971fd693a6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:14.419336 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 604/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:18:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:14.425261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d179ad7-b9d9-45db-981d-0fd6adbd5fdd None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:14.453135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:14.526079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d179ad7-b9d9-45db-981d-0fd6adbd5fdd None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:14.558576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990303a-84e7-41ca-a9d4-e5fc24b557e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:14.559433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990303a-84e7-41ca-a9d4-e5fc24b557e2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:14.559718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990303a-84e7-41ca-a9d4-e5fc24b557e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:14.559995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990303a-84e7-41ca-a9d4-e5fc24b557e2 None admin] PATH_INFO: `/v3/users/929471008a954911bfa3f0a0bfa7042e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:14.573308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990303a-84e7-41ca-a9d4-e5fc24b557e2 None admin] RBAC: Authorizing `identity:get_user(user_id=929471008a954911bfa3f0a0bfa7042e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:14.574754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990303a-84e7-41ca-a9d4-e5fc24b557e2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:14.576387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 578/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:18:14 2022] GET /identity/v3/users/929471008a954911bfa3f0a0bfa7042e => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:14.582074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6b8e-3576-4d86-a791-1e971fd693a6 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:14.637147 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:14.679329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6b8e-3576-4d86-a791-1e971fd693a6 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:14.712285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e5cfa-8826-4fa3-b0a5-2912209767db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:14.712951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e5cfa-8826-4fa3-b0a5-2912209767db None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:14.713239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e5cfa-8826-4fa3-b0a5-2912209767db None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:14.713518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e5cfa-8826-4fa3-b0a5-2912209767db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:14.778263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e5cfa-8826-4fa3-b0a5-2912209767db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:14.779647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e5cfa-8826-4fa3-b0a5-2912209767db None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:14.833748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:14.876638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 605/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:18:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:18:14.882514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990303a-84e7-41ca-a9d4-e5fc24b557e2 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:14.964297 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:14.980160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990303a-84e7-41ca-a9d4-e5fc24b557e2 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:15.013773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee7764-0b27-42eb-b6c9-b197014e1486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:15.014615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7764-0b27-42eb-b6c9-b197014e1486 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:15.014893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7764-0b27-42eb-b6c9-b197014e1486 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:15.015163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7764-0b27-42eb-b6c9-b197014e1486 None admin] PATH_INFO: `/v3/projects/c2ad285052d1409c8f0c13eb967ee4a5` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:15.016319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee7764-0b27-42eb-b6c9-b197014e1486 None admin] RBAC: Authorizing `identity:get_project(project_id=c2ad285052d1409c8f0c13eb967ee4a5)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:15.017788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee7764-0b27-42eb-b6c9-b197014e1486 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:15.019460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 579/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:18:14 2022] GET /identity/v3/projects/c2ad285052d1409c8f0c13eb967ee4a5 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:15.025859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e5cfa-8826-4fa3-b0a5-2912209767db None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:15.120569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e5cfa-8826-4fa3-b0a5-2912209767db None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:15.153644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e9962-14fa-4255-b892-d8d4f8a16682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:15.154288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e9962-14fa-4255-b892-d8d4f8a16682 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:15.154568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e9962-14fa-4255-b892-d8d4f8a16682 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:15.154843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e9962-14fa-4255-b892-d8d4f8a16682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:15.155663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e9962-14fa-4255-b892-d8d4f8a16682 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=c2ad285052d1409c8f0c13eb967ee4a5, user.id=929471008a954911bfa3f0a0bfa7042e) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:18:15.155973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e9962-14fa-4255-b892-d8d4f8a16682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:15.157427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e9962-14fa-4255-b892-d8d4f8a16682 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:15.162658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 606/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:18:15 2022] GET /identity/v3/role_assignments?user.id=929471008a954911bfa3f0a0bfa7042e&scope.project.id=c2ad285052d1409c8f0c13eb967ee4a5&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:17.330728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cc0c5-126d-4a80-95b2-effb34cdd9a7 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:17.331292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cc0c5-126d-4a80-95b2-effb34cdd9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:17.331292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cc0c5-126d-4a80-95b2-effb34cdd9a7 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:17.332635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 580/1186] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:18:17.338596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee7497-3d1c-4e45-aa21-709d17798d4c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:17.339003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee7497-3d1c-4e45-aa21-709d17798d4c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:17.339121 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee7497-3d1c-4e45-aa21-709d17798d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:17.355907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d6ee7497-3d1c-4e45-aa21-709d17798d4c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:18:17.357125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d6ee7497-3d1c-4e45-aa21-709d17798d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:18:17.370724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:17.422101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 607/1187] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:18:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:18:17.444132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cc0c5-126d-4a80-95b2-effb34cdd9a7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:17.542704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cc0c5-126d-4a80-95b2-effb34cdd9a7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:17.575620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942c10b-c52f-44f6-bcea-b5916eeb5170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:17.576391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942c10b-c52f-44f6-bcea-b5916eeb5170 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:17.576670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942c10b-c52f-44f6-bcea-b5916eeb5170 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:17.576969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942c10b-c52f-44f6-bcea-b5916eeb5170 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:17.590709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942c10b-c52f-44f6-bcea-b5916eeb5170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-6942c10b-c52f-44f6-bcea-b5916eeb5170 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:18:17.598263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:18:17.599356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:18:17.600559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 581/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:18:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:18:17.605456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee7497-3d1c-4e45-aa21-709d17798d4c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:17.699991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee7497-3d1c-4e45-aa21-709d17798d4c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:17.731664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e62d9-5dd6-4ca0-b9b5-48eb5c6a156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:17.732320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e62d9-5dd6-4ca0-b9b5-48eb5c6a156b None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:17.732602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e62d9-5dd6-4ca0-b9b5-48eb5c6a156b None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:17.732899 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e62d9-5dd6-4ca0-b9b5-48eb5c6a156b None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:17.733641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e62d9-5dd6-4ca0-b9b5-48eb5c6a156b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:18:17.733947 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e62d9-5dd6-4ca0-b9b5-48eb5c6a156b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:17.735294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e62d9-5dd6-4ca0-b9b5-48eb5c6a156b None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:17.743553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 608/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:18:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:17.749395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942c10b-c52f-44f6-bcea-b5916eeb5170 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:17.765288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:17.847921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942c10b-c52f-44f6-bcea-b5916eeb5170 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:17.880507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cddd9c-eb94-4cfd-bbb6-eb2f534316e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:17.881380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cddd9c-eb94-4cfd-bbb6-eb2f534316e9 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:17.881657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cddd9c-eb94-4cfd-bbb6-eb2f534316e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:17.881926 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cddd9c-eb94-4cfd-bbb6-eb2f534316e9 None admin] PATH_INFO: `/v3/users/929471008a954911bfa3f0a0bfa7042e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:17.883627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cddd9c-eb94-4cfd-bbb6-eb2f534316e9 None admin] RBAC: Authorizing `identity:get_user(user_id=929471008a954911bfa3f0a0bfa7042e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:17.885083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cddd9c-eb94-4cfd-bbb6-eb2f534316e9 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:17.886674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 582/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:18:17 2022] GET /identity/v3/users/929471008a954911bfa3f0a0bfa7042e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:17.892430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e62d9-5dd6-4ca0-b9b5-48eb5c6a156b None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:17.907372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:17.989407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e62d9-5dd6-4ca0-b9b5-48eb5c6a156b None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:18.021688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35eda9-6466-4b63-a775-03ba3e25b4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:18.022531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35eda9-6466-4b63-a775-03ba3e25b4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:18.022810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35eda9-6466-4b63-a775-03ba3e25b4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:18.023086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35eda9-6466-4b63-a775-03ba3e25b4e4 None admin] PATH_INFO: `/v3/projects/c2ad285052d1409c8f0c13eb967ee4a5` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:18.024290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35eda9-6466-4b63-a775-03ba3e25b4e4 None admin] RBAC: Authorizing `identity:get_project(project_id=c2ad285052d1409c8f0c13eb967ee4a5)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:18.025722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35eda9-6466-4b63-a775-03ba3e25b4e4 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:18.027380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 609/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:18:17 2022] GET /identity/v3/projects/c2ad285052d1409c8f0c13eb967ee4a5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:18.033087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cddd9c-eb94-4cfd-bbb6-eb2f534316e9 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:18.110555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:18.129663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cddd9c-eb94-4cfd-bbb6-eb2f534316e9 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:18.161325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7608d27-16e8-420d-b508-d14d63a39595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:18.162189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7608d27-16e8-420d-b508-d14d63a39595 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:18.162465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7608d27-16e8-420d-b508-d14d63a39595 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:18.162732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7608d27-16e8-420d-b508-d14d63a39595 None admin] PATH_INFO: `/v3/projects/c2ad285052d1409c8f0c13eb967ee4a5/users/929471008a954911bfa3f0a0bfa7042e/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:18.165418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7608d27-16e8-420d-b508-d14d63a39595 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2ad285052d1409c8f0c13eb967ee4a5, user_id=929471008a954911bfa3f0a0bfa7042e, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:18.166932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7608d27-16e8-420d-b508-d14d63a39595 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:18.183525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 583/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:18:18 2022] PUT /identity/v3/projects/c2ad285052d1409c8f0c13eb967ee4a5/users/929471008a954911bfa3f0a0bfa7042e/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:18:20.383976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df6ea9-2c82-4771-96cd-9bf7044d021b None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:20.384528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df6ea9-2c82-4771-96cd-9bf7044d021b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:20.384528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df6ea9-2c82-4771-96cd-9bf7044d021b None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:20.385920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 610/1193] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:18:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:18:20.391193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f67a6-afb9-4f4b-985f-0e73c3349a23 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:20.391597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f67a6-afb9-4f4b-985f-0e73c3349a23 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:20.391752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f67a6-afb9-4f4b-985f-0e73c3349a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:20.408917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-2f9f67a6-afb9-4f4b-985f-0e73c3349a23 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:18:20.410104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-2f9f67a6-afb9-4f4b-985f-0e73c3349a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:18:20.474499 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 584/1194] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:18:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:18:20.497624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df6ea9-2c82-4771-96cd-9bf7044d021b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:20.577762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:20.597320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df6ea9-2c82-4771-96cd-9bf7044d021b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:20.630446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c2013-c396-4d5c-b563-9483e3c87d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:20.631237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c2013-c396-4d5c-b563-9483e3c87d38 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:20.631516 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c2013-c396-4d5c-b563-9483e3c87d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:20.631788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c2013-c396-4d5c-b563-9483e3c87d38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:20.642784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c2013-c396-4d5c-b563-9483e3c87d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-590c2013-c396-4d5c-b563-9483e3c87d38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:18:20.650489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:18:20.651570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:18:20.652827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 611/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:18:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:18:20.657681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f67a6-afb9-4f4b-985f-0e73c3349a23 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:20.724313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:20.755523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f67a6-afb9-4f4b-985f-0e73c3349a23 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:20.788352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719c28f-e882-4f0a-9f9a-7d32a7e92ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:20.789026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719c28f-e882-4f0a-9f9a-7d32a7e92ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:20.789308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719c28f-e882-4f0a-9f9a-7d32a7e92ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:20.789581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719c28f-e882-4f0a-9f9a-7d32a7e92ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:20.790321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719c28f-e882-4f0a-9f9a-7d32a7e92ff0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:18:20.790625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719c28f-e882-4f0a-9f9a-7d32a7e92ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:20.791994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719c28f-e882-4f0a-9f9a-7d32a7e92ff0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:20.799882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 585/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:18:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:20.806343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c2013-c396-4d5c-b563-9483e3c87d38 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:20.911481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c2013-c396-4d5c-b563-9483e3c87d38 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:20.944685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad0c30-b59f-47b7-a6df-0bca028dba98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:20.945575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad0c30-b59f-47b7-a6df-0bca028dba98 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:20.945858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad0c30-b59f-47b7-a6df-0bca028dba98 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:20.946131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad0c30-b59f-47b7-a6df-0bca028dba98 None admin] PATH_INFO: `/v3/users/929471008a954911bfa3f0a0bfa7042e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:20.947305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad0c30-b59f-47b7-a6df-0bca028dba98 None admin] RBAC: Authorizing `identity:get_user(user_id=929471008a954911bfa3f0a0bfa7042e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:20.948742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad0c30-b59f-47b7-a6df-0bca028dba98 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:20.950387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 612/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Mon May 23 15:18:20 2022] GET /identity/v3/users/929471008a954911bfa3f0a0bfa7042e => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:20.956133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719c28f-e882-4f0a-9f9a-7d32a7e92ff0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:21.055129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719c28f-e882-4f0a-9f9a-7d32a7e92ff0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:21.082421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:21.087753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64e3c2-5dec-4ed2-966f-8380e6d72d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:21.088400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e3c2-5dec-4ed2-966f-8380e6d72d54 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:21.088684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e3c2-5dec-4ed2-966f-8380e6d72d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:21.088993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e3c2-5dec-4ed2-966f-8380e6d72d54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:21.158882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64e3c2-5dec-4ed2-966f-8380e6d72d54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:21.160358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64e3c2-5dec-4ed2-966f-8380e6d72d54 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:21.256113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 586/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:18:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:18:21.262040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad0c30-b59f-47b7-a6df-0bca028dba98 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:21.272484 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:21.361523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad0c30-b59f-47b7-a6df-0bca028dba98 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:21.394291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073dc41-e00e-4e29-b55b-ceed0070a263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:21.395144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073dc41-e00e-4e29-b55b-ceed0070a263 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:21.395425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073dc41-e00e-4e29-b55b-ceed0070a263 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:21.395697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073dc41-e00e-4e29-b55b-ceed0070a263 None admin] PATH_INFO: `/v3/projects/c2ad285052d1409c8f0c13eb967ee4a5` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:21.396915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073dc41-e00e-4e29-b55b-ceed0070a263 None admin] RBAC: Authorizing `identity:get_project(project_id=c2ad285052d1409c8f0c13eb967ee4a5)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:21.399041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073dc41-e00e-4e29-b55b-ceed0070a263 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:21.400724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 613/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Mon May 23 15:18:21 2022] GET /identity/v3/projects/c2ad285052d1409c8f0c13eb967ee4a5 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:21.406630 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64e3c2-5dec-4ed2-966f-8380e6d72d54 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:18:21.444116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:21.512180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64e3c2-5dec-4ed2-966f-8380e6d72d54 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:18:21.545469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e130cf-8858-4144-9e06-b09a2f3d02a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:18:21.546116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e130cf-8858-4144-9e06-b09a2f3d02a1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:18:21.546397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e130cf-8858-4144-9e06-b09a2f3d02a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:18:21.546672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e130cf-8858-4144-9e06-b09a2f3d02a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:18:21.547500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e130cf-8858-4144-9e06-b09a2f3d02a1 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=c2ad285052d1409c8f0c13eb967ee4a5, user.id=929471008a954911bfa3f0a0bfa7042e) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:18:21.547811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e130cf-8858-4144-9e06-b09a2f3d02a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:18:21.549267 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e130cf-8858-4144-9e06-b09a2f3d02a1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:18:21.554984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 587/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:18:21 2022] GET /identity/v3/role_assignments?user.id=929471008a954911bfa3f0a0bfa7042e&scope.project.id=c2ad285052d1409c8f0c13eb967ee4a5&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:18:31.390630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=62 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:18:31.552383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=52 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:22.385599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f799f3f-38af-4094-b76c-929825ed59d6 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:22.386249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f799f3f-38af-4094-b76c-929825ed59d6 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:22.386371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f799f3f-38af-4094-b76c-929825ed59d6 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:22.388155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 614/1201] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:22.395258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634da0-f4bd-46eb-9071-64d796a401d4 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:22.395841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634da0-f4bd-46eb-9071-64d796a401d4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:22.395841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634da0-f4bd-46eb-9071-64d796a401d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:22.415350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-5e634da0-f4bd-46eb-9071-64d796a401d4 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:22.417119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5e634da0-f4bd-46eb-9071-64d796a401d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:22.512784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 588/1202] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:19:22.539054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f799f3f-38af-4094-b76c-929825ed59d6 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:22.670326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f799f3f-38af-4094-b76c-929825ed59d6 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:22.710470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f96a0-09e4-4eb5-86cc-0370d91ceca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:22.711270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f96a0-09e4-4eb5-86cc-0370d91ceca3 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:22.711594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f96a0-09e4-4eb5-86cc-0370d91ceca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:22.711869 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f96a0-09e4-4eb5-86cc-0370d91ceca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:22.725245 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f96a0-09e4-4eb5-86cc-0370d91ceca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:22.727106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f96a0-09e4-4eb5-86cc-0370d91ceca3 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-2b2f96a0-09e4-4eb5-86cc-0370d91ceca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:22.750565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:22.752400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:22.753480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:22.753480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:22.753480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:22.753480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:22.753480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 615/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:22.758926 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634da0-f4bd-46eb-9071-64d796a401d4 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:22.888526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634da0-f4bd-46eb-9071-64d796a401d4 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:22.922315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:22.929032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05623b-496e-4d96-96dc-7d2b574000fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:22.929768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05623b-496e-4d96-96dc-7d2b574000fe None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:22.930040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05623b-496e-4d96-96dc-7d2b574000fe None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:22.930274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05623b-496e-4d96-96dc-7d2b574000fe None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:22.931036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05623b-496e-4d96-96dc-7d2b574000fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:22.931313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05623b-496e-4d96-96dc-7d2b574000fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:22.932687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05623b-496e-4d96-96dc-7d2b574000fe None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:22.951193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 589/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:19:22.957637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f96a0-09e4-4eb5-86cc-0370d91ceca3 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:23.027342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:23.084902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f96a0-09e4-4eb5-86cc-0370d91ceca3 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:23.121646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fb0b0-bcd1-4a0e-9a97-8d3c8037d8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:23.122326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fb0b0-bcd1-4a0e-9a97-8d3c8037d8ef None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:23.122610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fb0b0-bcd1-4a0e-9a97-8d3c8037d8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:23.122887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fb0b0-bcd1-4a0e-9a97-8d3c8037d8ef None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:23.123589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fb0b0-bcd1-4a0e-9a97-8d3c8037d8ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:23.129426 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fb0b0-bcd1-4a0e-9a97-8d3c8037d8ef None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:23.151589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 616/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Mon May 23 15:19:22 2022] POST /identity/v3/users => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:19:25.723656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e7d9-edbd-4f35-90fa-baefd22876a0 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:25.724197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e7d9-edbd-4f35-90fa-baefd22876a0 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:25.724197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e7d9-edbd-4f35-90fa-baefd22876a0 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:25.725795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 590/1206] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:25.730951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771afb1d-063e-4683-b9a2-d94718b6af7b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:25.731418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771afb1d-063e-4683-b9a2-d94718b6af7b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:25.731514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771afb1d-063e-4683-b9a2-d94718b6af7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:25.753250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-771afb1d-063e-4683-b9a2-d94718b6af7b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:25.754540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-771afb1d-063e-4683-b9a2-d94718b6af7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:25.828335 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 617/1207] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:19:25.855466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471e7d9-edbd-4f35-90fa-baefd22876a0 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:25.977216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471e7d9-edbd-4f35-90fa-baefd22876a0 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:26.019314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8697ea8-73db-4b33-9ae5-147fb8e95c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:26.020221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8697ea8-73db-4b33-9ae5-147fb8e95c72 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:26.020729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8697ea8-73db-4b33-9ae5-147fb8e95c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:26.021247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8697ea8-73db-4b33-9ae5-147fb8e95c72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:26.038069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8697ea8-73db-4b33-9ae5-147fb8e95c72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-a8697ea8-73db-4b33-9ae5-147fb8e95c72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:19:26.047963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:26.053991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:26.055163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 591/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:19:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:26.055882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771afb1d-063e-4683-b9a2-d94718b6af7b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:26.099825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:26.173260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771afb1d-063e-4683-b9a2-d94718b6af7b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:26.214749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4ce6e-9847-4e0a-9c72-359eb8f0eb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:26.215543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4ce6e-9847-4e0a-9c72-359eb8f0eb7a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:26.215952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4ce6e-9847-4e0a-9c72-359eb8f0eb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:26.216232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4ce6e-9847-4e0a-9c72-359eb8f0eb7a None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:26.217271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4ce6e-9847-4e0a-9c72-359eb8f0eb7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:26.217707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4ce6e-9847-4e0a-9c72-359eb8f0eb7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:26.219261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4ce6e-9847-4e0a-9c72-359eb8f0eb7a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:26.236033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 618/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:19:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:26.241591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8697ea8-73db-4b33-9ae5-147fb8e95c72 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:26.303701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:26.366331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8697ea8-73db-4b33-9ae5-147fb8e95c72 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:26.406985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88185907-2b3e-4db9-90ab-27b34b111a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:26.408634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88185907-2b3e-4db9-90ab-27b34b111a2c None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:26.409009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88185907-2b3e-4db9-90ab-27b34b111a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:26.409329 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88185907-2b3e-4db9-90ab-27b34b111a2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:26.421103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88185907-2b3e-4db9-90ab-27b34b111a2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:26.422568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88185907-2b3e-4db9-90ab-27b34b111a2c None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-88185907-2b3e-4db9-90ab-27b34b111a2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:26.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:26.442574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:26.448065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:26.448065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:26.448065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:26.448065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:26.448065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 592/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:19:26.448420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4ce6e-9847-4e0a-9c72-359eb8f0eb7a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:26.574985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:26.577896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4ce6e-9847-4e0a-9c72-359eb8f0eb7a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:26.622144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1208639-3306-4a62-93c9-966517e75f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:26.622835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1208639-3306-4a62-93c9-966517e75f99 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:26.623280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1208639-3306-4a62-93c9-966517e75f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:26.623610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1208639-3306-4a62-93c9-966517e75f99 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:26.624417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1208639-3306-4a62-93c9-966517e75f99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:26.624725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1208639-3306-4a62-93c9-966517e75f99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:26.626475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1208639-3306-4a62-93c9-966517e75f99 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:26.641210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 619/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:26.647793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88185907-2b3e-4db9-90ab-27b34b111a2c None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:26.761018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88185907-2b3e-4db9-90ab-27b34b111a2c None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:26.762796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:26.799744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c1d70-4a47-41c5-9e97-f2e9b1ab66d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:26.800557 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c1d70-4a47-41c5-9e97-f2e9b1ab66d1 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:26.800941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c1d70-4a47-41c5-9e97-f2e9b1ab66d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:26.801231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c1d70-4a47-41c5-9e97-f2e9b1ab66d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:26.809134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c1d70-4a47-41c5-9e97-f2e9b1ab66d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:26.810844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c1d70-4a47-41c5-9e97-f2e9b1ab66d1 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-479c1d70-4a47-41c5-9e97-f2e9b1ab66d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:19:26.830688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:26.831729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:26.832698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 593/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:26.838469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1208639-3306-4a62-93c9-966517e75f99 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:26.960197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1208639-3306-4a62-93c9-966517e75f99 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:26.999050 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf44097-065d-47c9-a23c-83c3a59a0614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:26.999798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf44097-065d-47c9-a23c-83c3a59a0614 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:27.000111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf44097-065d-47c9-a23c-83c3a59a0614 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:27.000372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf44097-065d-47c9-a23c-83c3a59a0614 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:27.002100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf44097-065d-47c9-a23c-83c3a59a0614 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:27.002394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf44097-065d-47c9-a23c-83c3a59a0614 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:27.004002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf44097-065d-47c9-a23c-83c3a59a0614 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:27.028753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 620/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Mon May 23 15:19:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:27.030110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c1d70-4a47-41c5-9e97-f2e9b1ab66d1 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:27.144712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c1d70-4a47-41c5-9e97-f2e9b1ab66d1 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:27.184653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c955b1f-bfce-47c0-9de4-6a94285e43d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:27.185470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c955b1f-bfce-47c0-9de4-6a94285e43d0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:27.185922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c955b1f-bfce-47c0-9de4-6a94285e43d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:27.186307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c955b1f-bfce-47c0-9de4-6a94285e43d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:27.222358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:27.270177 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c955b1f-bfce-47c0-9de4-6a94285e43d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:27.271944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c955b1f-bfce-47c0-9de4-6a94285e43d0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:27.397893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 594/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:19:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:19:27.403630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf44097-065d-47c9-a23c-83c3a59a0614 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:27.425057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:27.524925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf44097-065d-47c9-a23c-83c3a59a0614 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:27.570312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7100057-ce74-4c16-b479-6e9d6d600492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:27.571263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100057-ce74-4c16-b479-6e9d6d600492 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:27.571634 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100057-ce74-4c16-b479-6e9d6d600492 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:27.572031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100057-ce74-4c16-b479-6e9d6d600492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:27.583360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7100057-ce74-4c16-b479-6e9d6d600492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:27.585299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7100057-ce74-4c16-b479-6e9d6d600492 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-e7100057-ce74-4c16-b479-6e9d6d600492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:27.619966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:27.621379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:27.622526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:27.622526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:27.622526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:27.622526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:27.630456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c955b1f-bfce-47c0-9de4-6a94285e43d0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:27.633048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 621/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:27.717187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:27.764523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c955b1f-bfce-47c0-9de4-6a94285e43d0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:27.808334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df6fc-d597-4b52-a56d-ffbfa3f87783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:27.809064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df6fc-d597-4b52-a56d-ffbfa3f87783 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:27.809388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df6fc-d597-4b52-a56d-ffbfa3f87783 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:27.809859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df6fc-d597-4b52-a56d-ffbfa3f87783 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:27.810860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df6fc-d597-4b52-a56d-ffbfa3f87783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:27.811271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df6fc-d597-4b52-a56d-ffbfa3f87783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:27.813089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df6fc-d597-4b52-a56d-ffbfa3f87783 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:27.832746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 595/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:27.834793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7100057-ce74-4c16-b479-6e9d6d600492 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:27.917528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:27.962592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7100057-ce74-4c16-b479-6e9d6d600492 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:28.004743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f6410-1bee-4e39-a9d4-0e5d1011404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:28.006096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f6410-1bee-4e39-a9d4-0e5d1011404d None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:28.006613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f6410-1bee-4e39-a9d4-0e5d1011404d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:28.006932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f6410-1bee-4e39-a9d4-0e5d1011404d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:28.016470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f6410-1bee-4e39-a9d4-0e5d1011404d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:28.019092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f6410-1bee-4e39-a9d4-0e5d1011404d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-d15f6410-1bee-4e39-a9d4-0e5d1011404d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:19:28.034749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:28.035698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:28.042306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:19:28.042306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:19:28.042306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:19:28.042306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:19:28.042306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:19:28.042306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:19:28.042306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:28.042306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:28.042306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 622/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:19:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:19:28.045247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df6fc-d597-4b52-a56d-ffbfa3f87783 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:28.171050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df6fc-d597-4b52-a56d-ffbfa3f87783 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:28.197982 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:28.211953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bf2e8-7c5a-4dc3-b52f-581118f9c3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:28.212702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bf2e8-7c5a-4dc3-b52f-581118f9c3eb None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:28.213038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bf2e8-7c5a-4dc3-b52f-581118f9c3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:28.213340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bf2e8-7c5a-4dc3-b52f-581118f9c3eb None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:28.214205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bf2e8-7c5a-4dc3-b52f-581118f9c3eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:28.214540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bf2e8-7c5a-4dc3-b52f-581118f9c3eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:28.216029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bf2e8-7c5a-4dc3-b52f-581118f9c3eb None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:28.229755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 596/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:28.236491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f6410-1bee-4e39-a9d4-0e5d1011404d None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:28.379044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f6410-1bee-4e39-a9d4-0e5d1011404d None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:28.426956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:28.434835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a35908-b30f-47df-a942-27d79b56c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:28.435501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a35908-b30f-47df-a942-27d79b56c305 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:28.435785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a35908-b30f-47df-a942-27d79b56c305 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:28.436061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a35908-b30f-47df-a942-27d79b56c305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:28.437148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a35908-b30f-47df-a942-27d79b56c305 None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=3a7dd5a0102d4d3493f09e809cb3caaf) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:28.437416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a35908-b30f-47df-a942-27d79b56c305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:28.439762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a35908-b30f-47df-a942-27d79b56c305 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:28.446860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 623/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:19:28 2022] GET /identity/v3/role_assignments?user.id=3a7dd5a0102d4d3493f09e809cb3caaf&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:19:30.711068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b38fe-10f2-4b8d-95ba-5ea066bb1778 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:30.711688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b38fe-10f2-4b8d-95ba-5ea066bb1778 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:30.711807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b38fe-10f2-4b8d-95ba-5ea066bb1778 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:30.713504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 597/1220] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:30.719174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95595f6-b7cb-499c-8dc4-4bb0f94c2b27 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:30.719809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95595f6-b7cb-499c-8dc4-4bb0f94c2b27 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:30.719809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95595f6-b7cb-499c-8dc4-4bb0f94c2b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:30.737917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-c95595f6-b7cb-499c-8dc4-4bb0f94c2b27 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:30.739134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-c95595f6-b7cb-499c-8dc4-4bb0f94c2b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:30.810529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 624/1221] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:19:30.839628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b38fe-10f2-4b8d-95ba-5ea066bb1778 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:30.950175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b38fe-10f2-4b8d-95ba-5ea066bb1778 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:30.986885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04038a9-88f1-45dc-a334-db095ced3eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:30.987760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04038a9-88f1-45dc-a334-db095ced3eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:30.988102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04038a9-88f1-45dc-a334-db095ced3eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:30.988374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04038a9-88f1-45dc-a334-db095ced3eb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:31.001406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04038a9-88f1-45dc-a334-db095ced3eb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-e04038a9-88f1-45dc-a334-db095ced3eb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:19:31.010645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:31.011842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:31.012667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 598/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:19:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:31.019050 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95595f6-b7cb-499c-8dc4-4bb0f94c2b27 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:31.129524 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95595f6-b7cb-499c-8dc4-4bb0f94c2b27 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:31.156024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:31.168314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82287c-4d09-4187-b026-d7a9445b8a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:31.169026 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82287c-4d09-4187-b026-d7a9445b8a84 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:31.169334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82287c-4d09-4187-b026-d7a9445b8a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:31.169610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82287c-4d09-4187-b026-d7a9445b8a84 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:31.170358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82287c-4d09-4187-b026-d7a9445b8a84 None admin] RBAC: Adding query filter params (name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:31.170706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82287c-4d09-4187-b026-d7a9445b8a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:31.172044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82287c-4d09-4187-b026-d7a9445b8a84 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:31.181348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 625/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:19:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:31.187355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04038a9-88f1-45dc-a334-db095ced3eb6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:31.231375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:31.301214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04038a9-88f1-45dc-a334-db095ced3eb6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:31.341432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441da47e-b93a-447f-b6a6-7bef392a91ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:31.342359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441da47e-b93a-447f-b6a6-7bef392a91ea None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:31.342665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441da47e-b93a-447f-b6a6-7bef392a91ea None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:31.343042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441da47e-b93a-447f-b6a6-7bef392a91ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:31.360703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441da47e-b93a-447f-b6a6-7bef392a91ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:31.363067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441da47e-b93a-447f-b6a6-7bef392a91ea None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-441da47e-b93a-447f-b6a6-7bef392a91ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:31.388523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:31.389540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:31.390433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:31.390433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:31.390433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:31.390433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:31.390433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 599/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:31.398203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82287c-4d09-4187-b026-d7a9445b8a84 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:31.510540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82287c-4d09-4187-b026-d7a9445b8a84 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:31.545159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c74eec-bb99-4be6-b805-06d5fad96310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:31.545830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c74eec-bb99-4be6-b805-06d5fad96310 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:31.546157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c74eec-bb99-4be6-b805-06d5fad96310 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:31.546477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c74eec-bb99-4be6-b805-06d5fad96310 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:31.547308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c74eec-bb99-4be6-b805-06d5fad96310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:31.547687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c74eec-bb99-4be6-b805-06d5fad96310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:31.549221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c74eec-bb99-4be6-b805-06d5fad96310 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:31.563662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 626/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:31.570796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441da47e-b93a-447f-b6a6-7bef392a91ea None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:31.679360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:31.687085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441da47e-b93a-447f-b6a6-7bef392a91ea None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:31.727449 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0743792-fbf9-4f18-9a86-139905979153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:31.728304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0743792-fbf9-4f18-9a86-139905979153 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:31.728624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0743792-fbf9-4f18-9a86-139905979153 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:31.728985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0743792-fbf9-4f18-9a86-139905979153 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:31.739101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0743792-fbf9-4f18-9a86-139905979153 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:31.740729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0743792-fbf9-4f18-9a86-139905979153 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-a0743792-fbf9-4f18-9a86-139905979153 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:19:31.755776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:31.757146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:31.758644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 600/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:31.763526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c74eec-bb99-4be6-b805-06d5fad96310 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:31.814037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:31.877197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c74eec-bb99-4be6-b805-06d5fad96310 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:31.914374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8481af-4164-4e11-93fc-2d1e27f60069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:31.915045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8481af-4164-4e11-93fc-2d1e27f60069 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:31.915340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8481af-4164-4e11-93fc-2d1e27f60069 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:31.915599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8481af-4164-4e11-93fc-2d1e27f60069 None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:31.916556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8481af-4164-4e11-93fc-2d1e27f60069 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:31.916875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8481af-4164-4e11-93fc-2d1e27f60069 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:31.918354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8481af-4164-4e11-93fc-2d1e27f60069 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:31.939457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 627/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Mon May 23 15:19:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:31.945640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0743792-fbf9-4f18-9a86-139905979153 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:32.050781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0743792-fbf9-4f18-9a86-139905979153 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:32.089678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187c2f0-1810-4e4d-b0f9-0df5bd0f2840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:32.090582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187c2f0-1810-4e4d-b0f9-0df5bd0f2840 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:32.090896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187c2f0-1810-4e4d-b0f9-0df5bd0f2840 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:32.091227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187c2f0-1810-4e4d-b0f9-0df5bd0f2840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:32.103500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187c2f0-1810-4e4d-b0f9-0df5bd0f2840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:32.105202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187c2f0-1810-4e4d-b0f9-0df5bd0f2840 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-c187c2f0-1810-4e4d-b0f9-0df5bd0f2840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:32.125094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:32.126520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:32.127753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:32.127753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:32.127753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:32.127753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:32.127753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 601/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:32.133078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8481af-4164-4e11-93fc-2d1e27f60069 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:32.250175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8481af-4164-4e11-93fc-2d1e27f60069 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:32.252972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:32.293144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df1a51-4c2f-43db-ab36-0c14e5069377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:32.293930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df1a51-4c2f-43db-ab36-0c14e5069377 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:32.294318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df1a51-4c2f-43db-ab36-0c14e5069377 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:32.294745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df1a51-4c2f-43db-ab36-0c14e5069377 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:32.295722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df1a51-4c2f-43db-ab36-0c14e5069377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:32.296064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df1a51-4c2f-43db-ab36-0c14e5069377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:32.297945 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df1a51-4c2f-43db-ab36-0c14e5069377 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:32.311611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 628/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:19:32.317638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187c2f0-1810-4e4d-b0f9-0df5bd0f2840 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:32.327302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:32.439573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187c2f0-1810-4e4d-b0f9-0df5bd0f2840 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:32.479455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cb144-2a3e-4957-a77a-3e134df8b390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:32.480346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cb144-2a3e-4957-a77a-3e134df8b390 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:32.480669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cb144-2a3e-4957-a77a-3e134df8b390 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:32.481036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cb144-2a3e-4957-a77a-3e134df8b390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:32.488291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cb144-2a3e-4957-a77a-3e134df8b390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:32.489850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cb144-2a3e-4957-a77a-3e134df8b390 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-1f3cb144-2a3e-4957-a77a-3e134df8b390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:19:32.506694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:32.507933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:32.509764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:19:32.509764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:19:32.509764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:19:32.509764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:19:32.509764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:19:32.509764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:19:32.509764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:32.509764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:32.509764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 602/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:19:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:32.514151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df1a51-4c2f-43db-ab36-0c14e5069377 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:32.633316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df1a51-4c2f-43db-ab36-0c14e5069377 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:32.670590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7fd6e-2c3d-496d-8525-112687e689d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:32.671416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7fd6e-2c3d-496d-8525-112687e689d1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:32.671713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7fd6e-2c3d-496d-8525-112687e689d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:32.672015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7fd6e-2c3d-496d-8525-112687e689d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:32.673636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7fd6e-2c3d-496d-8525-112687e689d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:32.673980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7fd6e-2c3d-496d-8525-112687e689d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:32.675543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7fd6e-2c3d-496d-8525-112687e689d1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:32.693368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 629/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:19:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:19:32.694675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cb144-2a3e-4957-a77a-3e134df8b390 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:32.771915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:32.810544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cb144-2a3e-4957-a77a-3e134df8b390 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:32.850343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18f093-57b8-4a38-9aa1-3ea4cae99973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:32.851396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18f093-57b8-4a38-9aa1-3ea4cae99973 None admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:32.851684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18f093-57b8-4a38-9aa1-3ea4cae99973 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:32.851966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18f093-57b8-4a38-9aa1-3ea4cae99973 None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/3a7dd5a0102d4d3493f09e809cb3caaf/roles/2c4cf4c4707947848a30121234a5157c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:32.868966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18f093-57b8-4a38-9aa1-3ea4cae99973 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=3a7dd5a0102d4d3493f09e809cb3caaf, role_id=2c4cf4c4707947848a30121234a5157c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:32.870352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18f093-57b8-4a38-9aa1-3ea4cae99973 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:32.884472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 603/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:19:32 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/3a7dd5a0102d4d3493f09e809cb3caaf/roles/2c4cf4c4707947848a30121234a5157c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:19:35.197717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17add6-5edb-4eae-b3dc-5cddc228439d None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:35.198365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17add6-5edb-4eae-b3dc-5cddc228439d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:35.198365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17add6-5edb-4eae-b3dc-5cddc228439d None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:35.199832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 630/1233] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:35.206104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446629ab-3e8b-41b5-a821-2c99269fbd3e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:35.206804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446629ab-3e8b-41b5-a821-2c99269fbd3e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:35.206804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446629ab-3e8b-41b5-a821-2c99269fbd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:35.226063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-446629ab-3e8b-41b5-a821-2c99269fbd3e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:35.227284 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-446629ab-3e8b-41b5-a821-2c99269fbd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:35.307309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 604/1234] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:19:35.334666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17add6-5edb-4eae-b3dc-5cddc228439d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:35.365571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:35.460860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17add6-5edb-4eae-b3dc-5cddc228439d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:35.504931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339c737-b9cd-47d4-a68e-ad982c4c9dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:35.505770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339c737-b9cd-47d4-a68e-ad982c4c9dcb None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:35.506197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339c737-b9cd-47d4-a68e-ad982c4c9dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:35.506686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339c737-b9cd-47d4-a68e-ad982c4c9dcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:35.519443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339c737-b9cd-47d4-a68e-ad982c4c9dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-c339c737-b9cd-47d4-a68e-ad982c4c9dcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:19:35.527304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:35.528531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:35.529657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 631/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Mon May 23 15:19:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:35.534674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446629ab-3e8b-41b5-a821-2c99269fbd3e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:35.587912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:35.650183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446629ab-3e8b-41b5-a821-2c99269fbd3e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:35.691542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716d89a-0c21-4076-b95b-6b00aeef4dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:35.692202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716d89a-0c21-4076-b95b-6b00aeef4dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:35.692537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716d89a-0c21-4076-b95b-6b00aeef4dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:35.692834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716d89a-0c21-4076-b95b-6b00aeef4dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:35.694047 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716d89a-0c21-4076-b95b-6b00aeef4dd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:35.694558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716d89a-0c21-4076-b95b-6b00aeef4dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:35.696330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716d89a-0c21-4076-b95b-6b00aeef4dd0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:35.707256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 605/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Mon May 23 15:19:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:35.713964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339c737-b9cd-47d4-a68e-ad982c4c9dcb None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:35.825547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:35.846474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339c737-b9cd-47d4-a68e-ad982c4c9dcb None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:35.883978 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2d71-79d2-4598-bfc3-19d71df59de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:35.884778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2d71-79d2-4598-bfc3-19d71df59de1 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:35.885084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2d71-79d2-4598-bfc3-19d71df59de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:35.885391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2d71-79d2-4598-bfc3-19d71df59de1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:35.896796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d2d71-79d2-4598-bfc3-19d71df59de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:35.898267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d2d71-79d2-4598-bfc3-19d71df59de1 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-360d2d71-79d2-4598-bfc3-19d71df59de1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:35.916831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:35.917972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:35.919096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:35.919096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:35.919096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:35.919096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:35.919096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 632/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:35.924473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716d89a-0c21-4076-b95b-6b00aeef4dd0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:36.038742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716d89a-0c21-4076-b95b-6b00aeef4dd0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:36.060044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:36.093847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ab27e-0daa-4dd1-8dd1-50f0f0d65168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:36.094926 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ab27e-0daa-4dd1-8dd1-50f0f0d65168 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:36.095397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ab27e-0daa-4dd1-8dd1-50f0f0d65168 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:36.095805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ab27e-0daa-4dd1-8dd1-50f0f0d65168 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:36.096847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ab27e-0daa-4dd1-8dd1-50f0f0d65168 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:36.097330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ab27e-0daa-4dd1-8dd1-50f0f0d65168 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:36.098820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ab27e-0daa-4dd1-8dd1-50f0f0d65168 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:36.115228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 606/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:36.122152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2d71-79d2-4598-bfc3-19d71df59de1 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:36.238431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2d71-79d2-4598-bfc3-19d71df59de1 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:36.282259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236628b0-f4ba-4967-8c85-94f2bce0e779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:36.283218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236628b0-f4ba-4967-8c85-94f2bce0e779 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:36.283508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236628b0-f4ba-4967-8c85-94f2bce0e779 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:36.283789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236628b0-f4ba-4967-8c85-94f2bce0e779 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:36.287862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:36.294316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236628b0-f4ba-4967-8c85-94f2bce0e779 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:36.295927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236628b0-f4ba-4967-8c85-94f2bce0e779 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-236628b0-f4ba-4967-8c85-94f2bce0e779 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:19:36.316353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:36.317623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:36.318714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 633/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:36.325006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ab27e-0daa-4dd1-8dd1-50f0f0d65168 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:36.460651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ab27e-0daa-4dd1-8dd1-50f0f0d65168 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:36.501418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00966d9c-5887-4c82-94f5-499ef607a349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:36.502286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00966d9c-5887-4c82-94f5-499ef607a349 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:36.502573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00966d9c-5887-4c82-94f5-499ef607a349 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:36.502852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00966d9c-5887-4c82-94f5-499ef607a349 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:36.503694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00966d9c-5887-4c82-94f5-499ef607a349 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:36.504003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00966d9c-5887-4c82-94f5-499ef607a349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:36.505480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00966d9c-5887-4c82-94f5-499ef607a349 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:36.525261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 607/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Mon May 23 15:19:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:36.531276 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236628b0-f4ba-4967-8c85-94f2bce0e779 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:36.657674 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236628b0-f4ba-4967-8c85-94f2bce0e779 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:36.694363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5f183-4b2b-4960-9ee5-47b5a9ba3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:36.695030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5f183-4b2b-4960-9ee5-47b5a9ba3713 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:36.695290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5f183-4b2b-4960-9ee5-47b5a9ba3713 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:36.695562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5f183-4b2b-4960-9ee5-47b5a9ba3713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:36.764749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:36.774159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5f183-4b2b-4960-9ee5-47b5a9ba3713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:36.775820 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5f183-4b2b-4960-9ee5-47b5a9ba3713 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:36.879678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 634/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:19:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:19:36.885444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00966d9c-5887-4c82-94f5-499ef607a349 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:36.918336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:37.006085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00966d9c-5887-4c82-94f5-499ef607a349 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:37.045421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03525dfd-7ffc-40a1-ab7b-f8db6a85f2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:37.046469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03525dfd-7ffc-40a1-ab7b-f8db6a85f2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:37.046854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03525dfd-7ffc-40a1-ab7b-f8db6a85f2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:37.047469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03525dfd-7ffc-40a1-ab7b-f8db6a85f2d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:37.059430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03525dfd-7ffc-40a1-ab7b-f8db6a85f2d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:37.060907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03525dfd-7ffc-40a1-ab7b-f8db6a85f2d0 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-03525dfd-7ffc-40a1-ab7b-f8db6a85f2d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:37.082714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:37.083715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:37.084601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:37.084601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:37.084601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:37.084601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:37.084996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 608/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:37.090703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5f183-4b2b-4960-9ee5-47b5a9ba3713 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:37.198056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:37.205110 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5f183-4b2b-4960-9ee5-47b5a9ba3713 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:37.244769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7197b-0864-467c-bea6-d46e29abbf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:37.245569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7197b-0864-467c-bea6-d46e29abbf23 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:37.246222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7197b-0864-467c-bea6-d46e29abbf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:37.246582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7197b-0864-467c-bea6-d46e29abbf23 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:37.247491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7197b-0864-467c-bea6-d46e29abbf23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:37.247906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7197b-0864-467c-bea6-d46e29abbf23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:37.249451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7197b-0864-467c-bea6-d46e29abbf23 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:37.269580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 635/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:37.270213 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03525dfd-7ffc-40a1-ab7b-f8db6a85f2d0 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:37.356849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:37.388893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03525dfd-7ffc-40a1-ab7b-f8db6a85f2d0 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:37.434838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86465566-b020-4029-8e32-c095015db453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:37.435788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86465566-b020-4029-8e32-c095015db453 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:37.436190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86465566-b020-4029-8e32-c095015db453 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:37.436505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86465566-b020-4029-8e32-c095015db453 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:37.445764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86465566-b020-4029-8e32-c095015db453 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:37.447515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86465566-b020-4029-8e32-c095015db453 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-86465566-b020-4029-8e32-c095015db453 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:19:37.462674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:37.463830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:37.464808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:19:37.464808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:19:37.464808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:19:37.464808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:19:37.464808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:19:37.464808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:19:37.464808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:37.464808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:37.464808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 609/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:37.473036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7197b-0864-467c-bea6-d46e29abbf23 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:37.596743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7197b-0864-467c-bea6-d46e29abbf23 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:37.640143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b3535-e088-479f-8db7-10e9831ba87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:37.640895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3535-e088-479f-8db7-10e9831ba87a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:37.641188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3535-e088-479f-8db7-10e9831ba87a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:37.641470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3535-e088-479f-8db7-10e9831ba87a None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:37.642526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b3535-e088-479f-8db7-10e9831ba87a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:37.642896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b3535-e088-479f-8db7-10e9831ba87a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:37.644678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b3535-e088-479f-8db7-10e9831ba87a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:37.665361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 636/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:37.666612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86465566-b020-4029-8e32-c095015db453 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:37.782425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86465566-b020-4029-8e32-c095015db453 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:37.823688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d54965-c32c-4915-856f-7022dc9febaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:37.824536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d54965-c32c-4915-856f-7022dc9febaf None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:37.824960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d54965-c32c-4915-856f-7022dc9febaf None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:37.825438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d54965-c32c-4915-856f-7022dc9febaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:37.826626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d54965-c32c-4915-856f-7022dc9febaf None admin] RBAC: Adding query filter params (role.id=2c4cf4c4707947848a30121234a5157c, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=3a7dd5a0102d4d3493f09e809cb3caaf) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:37.826943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d54965-c32c-4915-856f-7022dc9febaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:37.829414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d54965-c32c-4915-856f-7022dc9febaf None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:37.836937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 610/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:19:37 2022] GET /identity/v3/role_assignments?user.id=3a7dd5a0102d4d3493f09e809cb3caaf&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2c4cf4c4707947848a30121234a5157c => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:40.254741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c338a-b7e0-4c94-819f-3367a0704302 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:40.255307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c338a-b7e0-4c94-819f-3367a0704302 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:40.255307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c338a-b7e0-4c94-819f-3367a0704302 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:40.256688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 637/1247] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:40.262775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caefaf3-9f32-4a32-b970-620a40ba4168 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:40.263653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caefaf3-9f32-4a32-b970-620a40ba4168 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:40.263653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caefaf3-9f32-4a32-b970-620a40ba4168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:40.269660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:40.283296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-0caefaf3-9f32-4a32-b970-620a40ba4168 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:40.284815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0caefaf3-9f32-4a32-b970-620a40ba4168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:40.364213 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 611/1248] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:19:40.391774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c338a-b7e0-4c94-819f-3367a0704302 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:40.418190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:40.525648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c338a-b7e0-4c94-819f-3367a0704302 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:40.564502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d65eb2-e3dd-4541-bcb0-b1df5ff5d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:40.565441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d65eb2-e3dd-4541-bcb0-b1df5ff5d487 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:40.565757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d65eb2-e3dd-4541-bcb0-b1df5ff5d487 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:40.566034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d65eb2-e3dd-4541-bcb0-b1df5ff5d487 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:40.579018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d65eb2-e3dd-4541-bcb0-b1df5ff5d487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-20d65eb2-e3dd-4541-bcb0-b1df5ff5d487 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:19:40.587966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:40.589357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:40.590566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 638/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:19:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:19:40.595659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caefaf3-9f32-4a32-b970-620a40ba4168 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:40.719009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caefaf3-9f32-4a32-b970-620a40ba4168 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:40.758365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:40.765138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaa733-04fb-43b2-9516-d85ece3bbdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:40.765909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaa733-04fb-43b2-9516-d85ece3bbdbe None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:40.766187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaa733-04fb-43b2-9516-d85ece3bbdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:40.766506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaa733-04fb-43b2-9516-d85ece3bbdbe None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:40.767426 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaa733-04fb-43b2-9516-d85ece3bbdbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:40.767803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaa733-04fb-43b2-9516-d85ece3bbdbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:40.770058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaa733-04fb-43b2-9516-d85ece3bbdbe None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:40.779044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 612/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:19:40.785834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d65eb2-e3dd-4541-bcb0-b1df5ff5d487 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:40.880807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:40.905040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d65eb2-e3dd-4541-bcb0-b1df5ff5d487 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:40.943128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c5b7d-a068-4ce1-8a25-910d692b0efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:40.943980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c5b7d-a068-4ce1-8a25-910d692b0efe None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:40.944285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c5b7d-a068-4ce1-8a25-910d692b0efe None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:40.944555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c5b7d-a068-4ce1-8a25-910d692b0efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:40.956005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c5b7d-a068-4ce1-8a25-910d692b0efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:40.957491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c5b7d-a068-4ce1-8a25-910d692b0efe None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-9e6c5b7d-a068-4ce1-8a25-910d692b0efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:40.975843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:40.976996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:40.978254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:40.978254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:40.978254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:40.978254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:40.978254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 639/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:19:40.983207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaa733-04fb-43b2-9516-d85ece3bbdbe None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:41.092536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaa733-04fb-43b2-9516-d85ece3bbdbe None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:41.129617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ee13b-3805-4320-8f2d-a022e336b56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:41.130411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ee13b-3805-4320-8f2d-a022e336b56b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:41.130760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ee13b-3805-4320-8f2d-a022e336b56b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:41.131033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ee13b-3805-4320-8f2d-a022e336b56b None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:41.131902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ee13b-3805-4320-8f2d-a022e336b56b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:41.132257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ee13b-3805-4320-8f2d-a022e336b56b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:41.133794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ee13b-3805-4320-8f2d-a022e336b56b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:41.148897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 613/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:41.152618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c5b7d-a068-4ce1-8a25-910d692b0efe None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:41.260369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c5b7d-a068-4ce1-8a25-910d692b0efe None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:41.288662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:41.297499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddc8e1-5a8e-4c21-be45-11d14a62a850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:41.298524 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddc8e1-5a8e-4c21-be45-11d14a62a850 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:41.298782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddc8e1-5a8e-4c21-be45-11d14a62a850 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:41.299064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddc8e1-5a8e-4c21-be45-11d14a62a850 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:41.306919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddc8e1-5a8e-4c21-be45-11d14a62a850 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:41.308395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddc8e1-5a8e-4c21-be45-11d14a62a850 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-1eddc8e1-5a8e-4c21-be45-11d14a62a850 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:19:41.323218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:41.324502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:41.325821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 640/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:41.330793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ee13b-3805-4320-8f2d-a022e336b56b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:41.397584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:41.454564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ee13b-3805-4320-8f2d-a022e336b56b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:41.497503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9b7a3-e306-40d4-ad5b-f54829b07b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:41.498207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9b7a3-e306-40d4-ad5b-f54829b07b52 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:41.498491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9b7a3-e306-40d4-ad5b-f54829b07b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:41.498766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9b7a3-e306-40d4-ad5b-f54829b07b52 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:41.499622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9b7a3-e306-40d4-ad5b-f54829b07b52 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:41.500515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9b7a3-e306-40d4-ad5b-f54829b07b52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:41.502192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9b7a3-e306-40d4-ad5b-f54829b07b52 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:41.528711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 614/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Mon May 23 15:19:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:41.534699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddc8e1-5a8e-4c21-be45-11d14a62a850 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:41.645723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddc8e1-5a8e-4c21-be45-11d14a62a850 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:41.681741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323fd1f-2414-4cd8-83e9-c880e09bea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:41.682329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323fd1f-2414-4cd8-83e9-c880e09bea1a None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:41.682615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323fd1f-2414-4cd8-83e9-c880e09bea1a None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:41.682893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323fd1f-2414-4cd8-83e9-c880e09bea1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:41.741705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:41.756090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323fd1f-2414-4cd8-83e9-c880e09bea1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:41.757611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323fd1f-2414-4cd8-83e9-c880e09bea1a None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:41.861235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 641/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:19:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:19:41.867328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9b7a3-e306-40d4-ad5b-f54829b07b52 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:41.975090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9b7a3-e306-40d4-ad5b-f54829b07b52 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:41.989027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:42.011744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171c19a-46be-4424-b09e-98e624a7c59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:42.012537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c19a-46be-4424-b09e-98e624a7c59e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:42.012816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c19a-46be-4424-b09e-98e624a7c59e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:42.013118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c19a-46be-4424-b09e-98e624a7c59e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:42.023655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171c19a-46be-4424-b09e-98e624a7c59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:42.025140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171c19a-46be-4424-b09e-98e624a7c59e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-3171c19a-46be-4424-b09e-98e624a7c59e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:42.042269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:42.043372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:42.044565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:42.044565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:42.044565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:42.044565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:42.044565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 615/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:42.050796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323fd1f-2414-4cd8-83e9-c880e09bea1a None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:42.142645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:42.154972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323fd1f-2414-4cd8-83e9-c880e09bea1a None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:42.189278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7c1ae-5c66-4986-9706-119f11356500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:42.189938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7c1ae-5c66-4986-9706-119f11356500 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:42.190222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7c1ae-5c66-4986-9706-119f11356500 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:42.190497 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7c1ae-5c66-4986-9706-119f11356500 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:42.191208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7c1ae-5c66-4986-9706-119f11356500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:42.191515 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7c1ae-5c66-4986-9706-119f11356500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:42.192881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7c1ae-5c66-4986-9706-119f11356500 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:42.204384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 642/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:42.210118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171c19a-46be-4424-b09e-98e624a7c59e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:42.307209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171c19a-46be-4424-b09e-98e624a7c59e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:42.340495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175df4ac-5db8-418a-b50f-74da60d91eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:42.341293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175df4ac-5db8-418a-b50f-74da60d91eff None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:42.341573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175df4ac-5db8-418a-b50f-74da60d91eff None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:42.341847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175df4ac-5db8-418a-b50f-74da60d91eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:42.349675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175df4ac-5db8-418a-b50f-74da60d91eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:42.351568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175df4ac-5db8-418a-b50f-74da60d91eff None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-175df4ac-5db8-418a-b50f-74da60d91eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:19:42.369840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:42.371247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:42.372306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:19:42.372306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:19:42.372306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:19:42.372306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:19:42.372306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:19:42.372306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:19:42.372306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:42.372306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:42.372306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 616/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:19:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:42.379069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7c1ae-5c66-4986-9706-119f11356500 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:42.483225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7c1ae-5c66-4986-9706-119f11356500 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:42.516855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd18db-679a-49e5-8815-dcf9594781da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:42.517534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd18db-679a-49e5-8815-dcf9594781da None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:42.517818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd18db-679a-49e5-8815-dcf9594781da None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:42.518097 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd18db-679a-49e5-8815-dcf9594781da None admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:42.518873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd18db-679a-49e5-8815-dcf9594781da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:42.519182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd18db-679a-49e5-8815-dcf9594781da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:42.520611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd18db-679a-49e5-8815-dcf9594781da None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:42.532251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 643/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:19:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:42.538215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175df4ac-5db8-418a-b50f-74da60d91eff None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:42.564236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:42.636985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175df4ac-5db8-418a-b50f-74da60d91eff None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:42.670360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1629f70-0887-4572-a107-ab7536d0e38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:42.670999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1629f70-0887-4572-a107-ab7536d0e38d None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:42.671276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1629f70-0887-4572-a107-ab7536d0e38d None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:42.671545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1629f70-0887-4572-a107-ab7536d0e38d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:42.672411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1629f70-0887-4572-a107-ab7536d0e38d None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=3a7dd5a0102d4d3493f09e809cb3caaf) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:42.672720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1629f70-0887-4572-a107-ab7536d0e38d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:42.674197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1629f70-0887-4572-a107-ab7536d0e38d None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:42.679488 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 617/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:19:42 2022] GET /identity/v3/role_assignments?user.id=3a7dd5a0102d4d3493f09e809cb3caaf&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:44.862090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99584b46-29a2-43b5-81e2-1c6809e5d721 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:44.862602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99584b46-29a2-43b5-81e2-1c6809e5d721 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:44.862602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99584b46-29a2-43b5-81e2-1c6809e5d721 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:44.863925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 644/1261] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:44.869252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e929c-a5c4-47ef-9bee-5412f11db7e2 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:44.869655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e929c-a5c4-47ef-9bee-5412f11db7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:44.869815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e929c-a5c4-47ef-9bee-5412f11db7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:44.887223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a75e929c-a5c4-47ef-9bee-5412f11db7e2 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:44.888372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a75e929c-a5c4-47ef-9bee-5412f11db7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:44.952511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 618/1262] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:19:44.975070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99584b46-29a2-43b5-81e2-1c6809e5d721 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:44.987471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:45.075647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99584b46-29a2-43b5-81e2-1c6809e5d721 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:45.108459 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c942cc-925a-478b-8c77-ebaa5f064f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:45.109391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c942cc-925a-478b-8c77-ebaa5f064f29 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:45.109667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c942cc-925a-478b-8c77-ebaa5f064f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:45.109941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c942cc-925a-478b-8c77-ebaa5f064f29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:45.119764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c942cc-925a-478b-8c77-ebaa5f064f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-86c942cc-925a-478b-8c77-ebaa5f064f29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:19:45.128239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:45.129321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:45.130256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 645/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:19:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:45.135387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e929c-a5c4-47ef-9bee-5412f11db7e2 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:45.153633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:45.232086 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e929c-a5c4-47ef-9bee-5412f11db7e2 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:45.265292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78311234-eb6a-4bb7-8fbb-2c23e32ff404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:45.266664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78311234-eb6a-4bb7-8fbb-2c23e32ff404 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:45.266949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78311234-eb6a-4bb7-8fbb-2c23e32ff404 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:45.267221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78311234-eb6a-4bb7-8fbb-2c23e32ff404 None admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:45.267962 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78311234-eb6a-4bb7-8fbb-2c23e32ff404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:45.268265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78311234-eb6a-4bb7-8fbb-2c23e32ff404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:45.269655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78311234-eb6a-4bb7-8fbb-2c23e32ff404 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:45.277598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 619/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:19:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:45.283403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c942cc-925a-478b-8c77-ebaa5f064f29 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:45.350103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:45.392467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c942cc-925a-478b-8c77-ebaa5f064f29 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:45.432928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614853d-b4e0-44bd-b3eb-497e9d2446cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:45.433596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614853d-b4e0-44bd-b3eb-497e9d2446cc None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:45.433897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614853d-b4e0-44bd-b3eb-497e9d2446cc None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:45.434206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614853d-b4e0-44bd-b3eb-497e9d2446cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:45.444623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614853d-b4e0-44bd-b3eb-497e9d2446cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:45.446073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614853d-b4e0-44bd-b3eb-497e9d2446cc None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-1614853d-b4e0-44bd-b3eb-497e9d2446cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:45.463085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:45.464102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:45.465068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:45.465068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:45.465068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:45.465068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:45.465068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 646/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:45.470653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78311234-eb6a-4bb7-8fbb-2c23e32ff404 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:45.551513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:45.571779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78311234-eb6a-4bb7-8fbb-2c23e32ff404 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:45.604845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e289e-9f99-4b0a-af5d-e7e54e5eb5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:45.605530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e289e-9f99-4b0a-af5d-e7e54e5eb5db None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:45.605810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e289e-9f99-4b0a-af5d-e7e54e5eb5db None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:45.606089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e289e-9f99-4b0a-af5d-e7e54e5eb5db None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:45.606809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e289e-9f99-4b0a-af5d-e7e54e5eb5db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:45.607114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e289e-9f99-4b0a-af5d-e7e54e5eb5db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:45.608485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e289e-9f99-4b0a-af5d-e7e54e5eb5db None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:45.620804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 620/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:45.627417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614853d-b4e0-44bd-b3eb-497e9d2446cc None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:45.723692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614853d-b4e0-44bd-b3eb-497e9d2446cc None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:45.742365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:45.756856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b906a-dc7e-477b-9425-646ea57cd593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:45.757672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b906a-dc7e-477b-9425-646ea57cd593 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:45.757953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b906a-dc7e-477b-9425-646ea57cd593 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:45.758225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b906a-dc7e-477b-9425-646ea57cd593 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:45.765981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b906a-dc7e-477b-9425-646ea57cd593 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:45.767404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b906a-dc7e-477b-9425-646ea57cd593 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-fa0b906a-dc7e-477b-9425-646ea57cd593 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:19:45.781317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:45.782408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:45.783747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 647/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:45.788181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e289e-9f99-4b0a-af5d-e7e54e5eb5db None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:45.885852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e289e-9f99-4b0a-af5d-e7e54e5eb5db None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:45.918957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd229b-6b06-48af-914b-a907e8ee5f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:45.919613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd229b-6b06-48af-914b-a907e8ee5f44 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:45.919895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd229b-6b06-48af-914b-a907e8ee5f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:45.920166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd229b-6b06-48af-914b-a907e8ee5f44 None admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:45.921029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd229b-6b06-48af-914b-a907e8ee5f44 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:45.921337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd229b-6b06-48af-914b-a907e8ee5f44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:45.922863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd229b-6b06-48af-914b-a907e8ee5f44 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:45.931267 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:45.940844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 621/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Mon May 23 15:19:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:45.946382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b906a-dc7e-477b-9425-646ea57cd593 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:46.043723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b906a-dc7e-477b-9425-646ea57cd593 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:46.076263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934496a-19ec-4bc1-a208-27ba91b45480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:46.077063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934496a-19ec-4bc1-a208-27ba91b45480 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:46.077344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934496a-19ec-4bc1-a208-27ba91b45480 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:46.077618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934496a-19ec-4bc1-a208-27ba91b45480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:46.088062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934496a-19ec-4bc1-a208-27ba91b45480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:46.089589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934496a-19ec-4bc1-a208-27ba91b45480 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-6934496a-19ec-4bc1-a208-27ba91b45480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:46.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:46.107502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:46.108552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:46.108552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:46.108552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:46.108552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:46.108552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 648/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:46.113928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd229b-6b06-48af-914b-a907e8ee5f44 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:46.211689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd229b-6b06-48af-914b-a907e8ee5f44 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:46.244573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cb33a-df01-4e2f-af74-b9e00eeb973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:46.245250 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cb33a-df01-4e2f-af74-b9e00eeb973e None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:46.245533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cb33a-df01-4e2f-af74-b9e00eeb973e None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:46.245809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cb33a-df01-4e2f-af74-b9e00eeb973e None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:46.246791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cb33a-df01-4e2f-af74-b9e00eeb973e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:46.247095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cb33a-df01-4e2f-af74-b9e00eeb973e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:46.248453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cb33a-df01-4e2f-af74-b9e00eeb973e None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:46.259922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 622/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:46.265549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934496a-19ec-4bc1-a208-27ba91b45480 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:46.295694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:46.368629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934496a-19ec-4bc1-a208-27ba91b45480 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:46.402336 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed54c7c-a492-4db2-a6b2-734b936deeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:46.403117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed54c7c-a492-4db2-a6b2-734b936deeab None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:46.403393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed54c7c-a492-4db2-a6b2-734b936deeab None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:46.403658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed54c7c-a492-4db2-a6b2-734b936deeab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:46.410609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed54c7c-a492-4db2-a6b2-734b936deeab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:46.412025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed54c7c-a492-4db2-a6b2-734b936deeab None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-8ed54c7c-a492-4db2-a6b2-734b936deeab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:19:46.426358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:46.427402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:46.428506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:19:46.428506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:19:46.428506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:19:46.428506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:19:46.428506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:19:46.428506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:19:46.428506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:46.428506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:46.428506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 649/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:19:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:46.433445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cb33a-df01-4e2f-af74-b9e00eeb973e None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:46.510522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:46.552613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cb33a-df01-4e2f-af74-b9e00eeb973e None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:46.585596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43a2d3-cd4b-49fd-9b46-bd4074f97d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:46.586252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43a2d3-cd4b-49fd-9b46-bd4074f97d01 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:46.586536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43a2d3-cd4b-49fd-9b46-bd4074f97d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:46.586833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43a2d3-cd4b-49fd-9b46-bd4074f97d01 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:46.587592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43a2d3-cd4b-49fd-9b46-bd4074f97d01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:46.587906 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43a2d3-cd4b-49fd-9b46-bd4074f97d01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:46.589381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43a2d3-cd4b-49fd-9b46-bd4074f97d01 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:46.601218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 623/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:19:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:46.607287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed54c7c-a492-4db2-a6b2-734b936deeab None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:46.695403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:46.704797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed54c7c-a492-4db2-a6b2-734b936deeab None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:46.736986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3cef3-a614-4443-88c9-596ad5bd412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:46.737854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3cef3-a614-4443-88c9-596ad5bd412d None admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:46.738129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3cef3-a614-4443-88c9-596ad5bd412d None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:46.738400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3cef3-a614-4443-88c9-596ad5bd412d None admin] PATH_INFO: `/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/3a7dd5a0102d4d3493f09e809cb3caaf/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:46.740331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3cef3-a614-4443-88c9-596ad5bd412d None admin] RBAC: Authorizing `identity:create_grant(project_id=8d84c37c0bf2446aa66ba9a9783c454a, user_id=3a7dd5a0102d4d3493f09e809cb3caaf, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:46.741768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3cef3-a614-4443-88c9-596ad5bd412d None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:46.754704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 650/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Mon May 23 15:19:46 2022] PUT /identity/v3/projects/8d84c37c0bf2446aa66ba9a9783c454a/users/3a7dd5a0102d4d3493f09e809cb3caaf/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:19:48.933077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a021e-fb5d-4f25-b13f-6c0028021fad None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:48.933579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a021e-fb5d-4f25-b13f-6c0028021fad None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:48.933579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a021e-fb5d-4f25-b13f-6c0028021fad None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:48.934938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 624/1274] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:48.940074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cf8eb-8345-428a-97e2-af01f681a529 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:48.940494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cf8eb-8345-428a-97e2-af01f681a529 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:48.940617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cf8eb-8345-428a-97e2-af01f681a529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:48.956992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-cb4cf8eb-8345-428a-97e2-af01f681a529 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:48.958152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-cb4cf8eb-8345-428a-97e2-af01f681a529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:49.023759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 651/1275] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:19:49.048163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a021e-fb5d-4f25-b13f-6c0028021fad None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:49.146429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a021e-fb5d-4f25-b13f-6c0028021fad None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:49.163687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:49.180326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6582322-118d-45ac-b0ac-741cf72f3619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:49.181135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582322-118d-45ac-b0ac-741cf72f3619 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:49.181415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582322-118d-45ac-b0ac-741cf72f3619 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:49.181685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582322-118d-45ac-b0ac-741cf72f3619 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:49.192087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6582322-118d-45ac-b0ac-741cf72f3619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-e6582322-118d-45ac-b0ac-741cf72f3619 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:19:49.199554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:49.200615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:49.201834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 625/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Mon May 23 15:19:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:49.206648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cf8eb-8345-428a-97e2-af01f681a529 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:49.284629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:49.303688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cf8eb-8345-428a-97e2-af01f681a529 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:49.336678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdc2e6-dfa4-40ce-baa3-4237ad24f7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:49.337363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdc2e6-dfa4-40ce-baa3-4237ad24f7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:49.337646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdc2e6-dfa4-40ce-baa3-4237ad24f7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:49.337921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdc2e6-dfa4-40ce-baa3-4237ad24f7f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:49.338654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdc2e6-dfa4-40ce-baa3-4237ad24f7f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:49.338959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdc2e6-dfa4-40ce-baa3-4237ad24f7f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:49.340312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdc2e6-dfa4-40ce-baa3-4237ad24f7f0 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:49.348459 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 652/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Mon May 23 15:19:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:49.354009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6582322-118d-45ac-b0ac-741cf72f3619 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:49.467116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6582322-118d-45ac-b0ac-741cf72f3619 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:49.500389 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c857b6-035a-4b35-898a-c0e106ce871b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:49.501188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c857b6-035a-4b35-898a-c0e106ce871b None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:49.501470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c857b6-035a-4b35-898a-c0e106ce871b None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:49.501742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c857b6-035a-4b35-898a-c0e106ce871b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:49.511838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c857b6-035a-4b35-898a-c0e106ce871b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:49.513328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c857b6-035a-4b35-898a-c0e106ce871b None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-08c857b6-035a-4b35-898a-c0e106ce871b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:49.529826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:49.531532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:49.532731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:49.532731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:49.532731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:49.532731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:49.532731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 626/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:49.537264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdc2e6-dfa4-40ce-baa3-4237ad24f7f0 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:49.642363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdc2e6-dfa4-40ce-baa3-4237ad24f7f0 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:49.678337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659dee1-f962-4c33-ab73-2d9bba450325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:49.678991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659dee1-f962-4c33-ab73-2d9bba450325 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:49.679272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659dee1-f962-4c33-ab73-2d9bba450325 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:49.679547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659dee1-f962-4c33-ab73-2d9bba450325 None admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:49.680258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659dee1-f962-4c33-ab73-2d9bba450325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:49.680563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659dee1-f962-4c33-ab73-2d9bba450325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:49.681947 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659dee1-f962-4c33-ab73-2d9bba450325 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:49.693875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 653/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:49.699462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c857b6-035a-4b35-898a-c0e106ce871b None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:49.729868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:49.797679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c857b6-035a-4b35-898a-c0e106ce871b None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:49.830743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153e6cc-d78f-4ade-8ddd-83d574e6b6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:49.831535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153e6cc-d78f-4ade-8ddd-83d574e6b6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:49.831818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153e6cc-d78f-4ade-8ddd-83d574e6b6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:49.832093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153e6cc-d78f-4ade-8ddd-83d574e6b6a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:49.839390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153e6cc-d78f-4ade-8ddd-83d574e6b6a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:49.840842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153e6cc-d78f-4ade-8ddd-83d574e6b6a6 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-0153e6cc-d78f-4ade-8ddd-83d574e6b6a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:19:49.857437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:49.860040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:49.861155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 627/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:49.865024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659dee1-f962-4c33-ab73-2d9bba450325 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:49.868801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:49.961995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659dee1-f962-4c33-ab73-2d9bba450325 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:49.995109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefecf0-91a4-4f21-9bc0-dc3141289a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:49.995710 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefecf0-91a4-4f21-9bc0-dc3141289a6f None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:49.996257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefecf0-91a4-4f21-9bc0-dc3141289a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:49.996536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefecf0-91a4-4f21-9bc0-dc3141289a6f None admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:49.997390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefecf0-91a4-4f21-9bc0-dc3141289a6f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:49.997700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefecf0-91a4-4f21-9bc0-dc3141289a6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:49.999108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefecf0-91a4-4f21-9bc0-dc3141289a6f None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:50.016517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 654/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Mon May 23 15:19:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:50.022353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153e6cc-d78f-4ade-8ddd-83d574e6b6a6 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:50.111295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:50.121346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153e6cc-d78f-4ade-8ddd-83d574e6b6a6 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:50.156704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8cd10-ed1d-4f03-a28f-1b3c251e70a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:50.157505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8cd10-ed1d-4f03-a28f-1b3c251e70a2 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:50.157789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8cd10-ed1d-4f03-a28f-1b3c251e70a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:50.158068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8cd10-ed1d-4f03-a28f-1b3c251e70a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:50.223425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8cd10-ed1d-4f03-a28f-1b3c251e70a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:50.224887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8cd10-ed1d-4f03-a28f-1b3c251e70a2 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:50.310404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:50.320730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 628/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Mon May 23 15:19:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:19:50.326701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefecf0-91a4-4f21-9bc0-dc3141289a6f None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:50.387563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:50.425078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefecf0-91a4-4f21-9bc0-dc3141289a6f None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:50.458606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca23fb-043b-4fd1-96f1-0d677c194123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:50.459408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca23fb-043b-4fd1-96f1-0d677c194123 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:50.459687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca23fb-043b-4fd1-96f1-0d677c194123 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:50.459957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca23fb-043b-4fd1-96f1-0d677c194123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:50.470383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca23fb-043b-4fd1-96f1-0d677c194123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:50.471811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca23fb-043b-4fd1-96f1-0d677c194123 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-27ca23fb-043b-4fd1-96f1-0d677c194123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:19:50.489374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:50.490436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:50.496588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:19:50.496588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:19:50.496588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:19:50.496588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:50.496588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 655/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:50.496976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8cd10-ed1d-4f03-a28f-1b3c251e70a2 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:50.594801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8cd10-ed1d-4f03-a28f-1b3c251e70a2 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:50.627802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335331ac-c75c-4559-8281-96d7de860b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:50.628456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335331ac-c75c-4559-8281-96d7de860b64 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:50.628737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335331ac-c75c-4559-8281-96d7de860b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:50.629044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335331ac-c75c-4559-8281-96d7de860b64 None admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:50.629764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335331ac-c75c-4559-8281-96d7de860b64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:50.630068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335331ac-c75c-4559-8281-96d7de860b64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:50.631448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335331ac-c75c-4559-8281-96d7de860b64 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:50.642817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 629/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Mon May 23 15:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:50.648356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca23fb-043b-4fd1-96f1-0d677c194123 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:50.744882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca23fb-043b-4fd1-96f1-0d677c194123 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:50.758613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:50.778785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd9e6f-525e-45d2-84d5-4ba7a691d451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:50.779561 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd9e6f-525e-45d2-84d5-4ba7a691d451 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:50.779840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd9e6f-525e-45d2-84d5-4ba7a691d451 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:50.780109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd9e6f-525e-45d2-84d5-4ba7a691d451 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:50.787090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd9e6f-525e-45d2-84d5-4ba7a691d451 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:50.788513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd9e6f-525e-45d2-84d5-4ba7a691d451 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application [None req-d5fd9e6f-525e-45d2-84d5-4ba7a691d451 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:19:50.801821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application with Lock( May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:50.802822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:50.803915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:19:50.803915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:19:50.803915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:19:50.803915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:19:50.803915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:19:50.803915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:19:50.803915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:19:50.803915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: ERROR keystone.server.flask.application May 23 15:19:50.803915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 656/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Mon May 23 15:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:50.808769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335331ac-c75c-4559-8281-96d7de860b64 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:50.857712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:50.906602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335331ac-c75c-4559-8281-96d7de860b64 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:50.939078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2e765-62fd-4ea4-8858-d9ae7b22ca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:50.939718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2e765-62fd-4ea4-8858-d9ae7b22ca11 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:50.939996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2e765-62fd-4ea4-8858-d9ae7b22ca11 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:50.940265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2e765-62fd-4ea4-8858-d9ae7b22ca11 None admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:50.941071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2e765-62fd-4ea4-8858-d9ae7b22ca11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:50.941382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2e765-62fd-4ea4-8858-d9ae7b22ca11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:50.942804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2e765-62fd-4ea4-8858-d9ae7b22ca11 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:50.954976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 630/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Mon May 23 15:19:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:50.961246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd9e6f-525e-45d2-84d5-4ba7a691d451 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:51.058172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd9e6f-525e-45d2-84d5-4ba7a691d451 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:51.091134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742b7b1-76b5-4e85-b98a-87cb0322fec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:51.091800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742b7b1-76b5-4e85-b98a-87cb0322fec0 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:51.092083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742b7b1-76b5-4e85-b98a-87cb0322fec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:51.092359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742b7b1-76b5-4e85-b98a-87cb0322fec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:51.093202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742b7b1-76b5-4e85-b98a-87cb0322fec0 None admin] RBAC: Adding query filter params (role.id=2ec004855b6f4bd5a778750ef8fdda50, scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a, user.id=3a7dd5a0102d4d3493f09e809cb3caaf) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:51.093512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742b7b1-76b5-4e85-b98a-87cb0322fec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:51.094951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742b7b1-76b5-4e85-b98a-87cb0322fec0 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:51.100728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 657/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Mon May 23 15:19:50 2022] GET /identity/v3/role_assignments?user.id=3a7dd5a0102d4d3493f09e809cb3caaf&scope.project.id=8d84c37c0bf2446aa66ba9a9783c454a&role.id=2ec004855b6f4bd5a778750ef8fdda50 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:19:53.335583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50df396-a4f8-4fb5-9a7d-9798a449742c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:53.336142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50df396-a4f8-4fb5-9a7d-9798a449742c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:53.336142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50df396-a4f8-4fb5-9a7d-9798a449742c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:53.337551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 631/1288] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:53.342805 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb4a6-76a0-48f3-b63d-bc6c7996e75c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:53.343776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb4a6-76a0-48f3-b63d-bc6c7996e75c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:53.343902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb4a6-76a0-48f3-b63d-bc6c7996e75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:53.362965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0c8bb4a6-76a0-48f3-b63d-bc6c7996e75c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:53.364142 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0c8bb4a6-76a0-48f3-b63d-bc6c7996e75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:53.395826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:53.436032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 658/1289] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:19:53.458541 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50df396-a4f8-4fb5-9a7d-9798a449742c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:53.558520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50df396-a4f8-4fb5-9a7d-9798a449742c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:53.569056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:53.593074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53282a2-aa40-48a7-84f7-92e507694b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:53.594510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53282a2-aa40-48a7-84f7-92e507694b13 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:53.594797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53282a2-aa40-48a7-84f7-92e507694b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:53.595074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53282a2-aa40-48a7-84f7-92e507694b13 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:53.595727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53282a2-aa40-48a7-84f7-92e507694b13 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:53.597133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53282a2-aa40-48a7-84f7-92e507694b13 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-e53282a2-aa40-48a7-84f7-92e507694b13 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:53.609923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:53.611193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:53.612161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 632/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:19:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:53.619716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb4a6-76a0-48f3-b63d-bc6c7996e75c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:53.715204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb4a6-76a0-48f3-b63d-bc6c7996e75c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:53.747959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d2b05-4455-4dce-9a16-1ff8a73b1635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:53.748612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2b05-4455-4dce-9a16-1ff8a73b1635 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:53.748914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2b05-4455-4dce-9a16-1ff8a73b1635 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:53.749193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2b05-4455-4dce-9a16-1ff8a73b1635 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:53.750624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d2b05-4455-4dce-9a16-1ff8a73b1635 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:53.750927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d2b05-4455-4dce-9a16-1ff8a73b1635 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:53.752273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d2b05-4455-4dce-9a16-1ff8a73b1635 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:53.756999 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 659/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Mon May 23 15:19:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:53.762814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53282a2-aa40-48a7-84f7-92e507694b13 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:53.865274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53282a2-aa40-48a7-84f7-92e507694b13 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:53.898306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec662b15-c3db-4b99-a4aa-f6c6a3582e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:53.898953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec662b15-c3db-4b99-a4aa-f6c6a3582e25 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:53.899235 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec662b15-c3db-4b99-a4aa-f6c6a3582e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:53.899509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec662b15-c3db-4b99-a4aa-f6c6a3582e25 None admin] PATH_INFO: `/v3/services` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:53.900217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec662b15-c3db-4b99-a4aa-f6c6a3582e25 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:53.900522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec662b15-c3db-4b99-a4aa-f6c6a3582e25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:53.901978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec662b15-c3db-4b99-a4aa-f6c6a3582e25 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:53.907020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 633/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Mon May 23 15:19:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:56.079075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8183d07-b268-4832-a277-7a4e7cf0e437 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:56.079621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8183d07-b268-4832-a277-7a4e7cf0e437 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:56.079621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8183d07-b268-4832-a277-7a4e7cf0e437 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:56.081033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 660/1293] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:56.086142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982eda-b9db-4d61-9a89-b30bbdfb3d07 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:56.086560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982eda-b9db-4d61-9a89-b30bbdfb3d07 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:56.086702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982eda-b9db-4d61-9a89-b30bbdfb3d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:56.103944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-46982eda-b9db-4d61-9a89-b30bbdfb3d07 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:56.105162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-46982eda-b9db-4d61-9a89-b30bbdfb3d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:56.133834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:56.177337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 634/1294] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:19:56.200075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8183d07-b268-4832-a277-7a4e7cf0e437 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:56.230347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:56.307932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8183d07-b268-4832-a277-7a4e7cf0e437 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:56.341164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a779c-f484-43d0-8c11-9e6b15c365de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:56.341762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a779c-f484-43d0-8c11-9e6b15c365de None admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:56.342047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a779c-f484-43d0-8c11-9e6b15c365de None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:56.342325 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a779c-f484-43d0-8c11-9e6b15c365de None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:56.343041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a779c-f484-43d0-8c11-9e6b15c365de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:56.344439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a779c-f484-43d0-8c11-9e6b15c365de None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:56.360056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 661/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Mon May 23 15:19:56 2022] POST /identity/v3/services => generated 246 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:19:58.568451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136961e1-7713-44e4-97f1-ed566515355c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:58.569019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136961e1-7713-44e4-97f1-ed566515355c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:58.569019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136961e1-7713-44e4-97f1-ed566515355c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:58.570417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 635/1296] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:19:58.575675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839f067-50a0-484f-8845-a02a9261f069 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:58.576130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839f067-50a0-484f-8845-a02a9261f069 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:58.576225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839f067-50a0-484f-8845-a02a9261f069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:58.594043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-8839f067-50a0-484f-8845-a02a9261f069 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:19:58.595236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8839f067-50a0-484f-8845-a02a9261f069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:19:58.666716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 662/1297] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:19:58 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:19:58.689316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136961e1-7713-44e4-97f1-ed566515355c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:58.786900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136961e1-7713-44e4-97f1-ed566515355c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:58.820256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81baff2e-9ca6-47b0-b177-da1a33d675d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:58.821012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81baff2e-9ca6-47b0-b177-da1a33d675d3 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:58.821295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81baff2e-9ca6-47b0-b177-da1a33d675d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:58.821568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81baff2e-9ca6-47b0-b177-da1a33d675d3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:58.822219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81baff2e-9ca6-47b0-b177-da1a33d675d3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:58.823592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81baff2e-9ca6-47b0-b177-da1a33d675d3 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-81baff2e-9ca6-47b0-b177-da1a33d675d3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:19:58.835914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:19:58.837097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:19:58.837984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 636/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:19:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:19:58.843744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839f067-50a0-484f-8845-a02a9261f069 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:58.871219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:58.951591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839f067-50a0-484f-8845-a02a9261f069 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:58.985212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c61c7-be77-4222-af0c-d18d169fdec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:58.985884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c61c7-be77-4222-af0c-d18d169fdec3 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:58.986172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c61c7-be77-4222-af0c-d18d169fdec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:58.986453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c61c7-be77-4222-af0c-d18d169fdec3 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:58.987158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c61c7-be77-4222-af0c-d18d169fdec3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:58.987467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c61c7-be77-4222-af0c-d18d169fdec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:58.988821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c61c7-be77-4222-af0c-d18d169fdec3 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:58.993896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 663/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Mon May 23 15:19:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:59.000287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81baff2e-9ca6-47b0-b177-da1a33d675d3 None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:59.031937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:59.100358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81baff2e-9ca6-47b0-b177-da1a33d675d3 None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:59.134219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a18549-2cf1-4b3e-965b-f7fa4742c952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:59.134864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a18549-2cf1-4b3e-965b-f7fa4742c952 None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:59.135144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a18549-2cf1-4b3e-965b-f7fa4742c952 None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:59.135417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a18549-2cf1-4b3e-965b-f7fa4742c952 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:59.136199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a18549-2cf1-4b3e-965b-f7fa4742c952 None admin] RBAC: Adding query filter params (interface=public, service_id=75559086978a40218d3c163070990753, region_id=RegionOne) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:19:59.136507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a18549-2cf1-4b3e-965b-f7fa4742c952 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:59.137901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a18549-2cf1-4b3e-965b-f7fa4742c952 None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:59.143422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 637/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Mon May 23 15:19:58 2022] GET /identity/v3/endpoints?service_id=75559086978a40218d3c163070990753&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:19:59.149073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c61c7-be77-4222-af0c-d18d169fdec3 None admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:19:59.256270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:19:59.275350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c61c7-be77-4222-af0c-d18d169fdec3 None admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:19:59.310890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fe8df-fbf2-456e-a72e-64fb703aa289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:19:59.311564 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fe8df-fbf2-456e-a72e-64fb703aa289 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:19:59.312376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fe8df-fbf2-456e-a72e-64fb703aa289 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:19:59.312756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fe8df-fbf2-456e-a72e-64fb703aa289 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:19:59.313636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fe8df-fbf2-456e-a72e-64fb703aa289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:19:59.315007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fe8df-fbf2-456e-a72e-64fb703aa289 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:19:59.322179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 664/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Mon May 23 15:19:59 2022] GET /identity/v3/services => generated 2191 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:20:01.564802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f1cc-928d-4de4-8d30-62e6e1b53dc7 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:01.565367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f1cc-928d-4de4-8d30-62e6e1b53dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:01.565367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f1cc-928d-4de4-8d30-62e6e1b53dc7 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:01.566735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 638/1302] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:01.571810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66e5a9-806f-480a-ab5d-e0bf7b3e9db1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:01.572217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66e5a9-806f-480a-ab5d-e0bf7b3e9db1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:01.572371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66e5a9-806f-480a-ab5d-e0bf7b3e9db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:01.589573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-9f66e5a9-806f-480a-ab5d-e0bf7b3e9db1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:01.590757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9f66e5a9-806f-480a-ab5d-e0bf7b3e9db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:01.656808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 665/1303] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:20:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:20:01.679995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f1cc-928d-4de4-8d30-62e6e1b53dc7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:01.772498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:01.779300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f1cc-928d-4de4-8d30-62e6e1b53dc7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:01.812077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa3ae7-22c1-4aaf-8009-23b8485339ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:01.812859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa3ae7-22c1-4aaf-8009-23b8485339ec None admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:01.813166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa3ae7-22c1-4aaf-8009-23b8485339ec None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:01.813440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa3ae7-22c1-4aaf-8009-23b8485339ec None admin] PATH_INFO: `/v3/services/placement` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:01.814080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa3ae7-22c1-4aaf-8009-23b8485339ec None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:01.815490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa3ae7-22c1-4aaf-8009-23b8485339ec None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application [None req-6faa3ae7-22c1-4aaf-8009-23b8485339ec None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:01.827207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:01.828934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application with Lock( May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: ERROR keystone.server.flask.application May 23 15:20:01.829844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 639/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Mon May 23 15:20:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:01.835176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66e5a9-806f-480a-ab5d-e0bf7b3e9db1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:01.917246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:01.941547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66e5a9-806f-480a-ab5d-e0bf7b3e9db1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:01.973985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551e575-9493-4bc1-a045-08091dd8d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:01.974643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551e575-9493-4bc1-a045-08091dd8d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:01.974927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551e575-9493-4bc1-a045-08091dd8d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:01.975205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551e575-9493-4bc1-a045-08091dd8d6a6 None admin] PATH_INFO: `/v3/services` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:01.975909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551e575-9493-4bc1-a045-08091dd8d6a6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:01.976214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551e575-9493-4bc1-a045-08091dd8d6a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:01.977606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551e575-9493-4bc1-a045-08091dd8d6a6 None admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:01.982682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 666/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Mon May 23 15:20:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:01.988961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa3ae7-22c1-4aaf-8009-23b8485339ec None admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:02.085098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa3ae7-22c1-4aaf-8009-23b8485339ec None admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:02.117048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d4498-87b4-4c3d-bcb9-7634ddfd45cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:02.117687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d4498-87b4-4c3d-bcb9-7634ddfd45cc None admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:02.117965 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d4498-87b4-4c3d-bcb9-7634ddfd45cc None admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:02.118516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d4498-87b4-4c3d-bcb9-7634ddfd45cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:02.119235 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d4498-87b4-4c3d-bcb9-7634ddfd45cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:02.120590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d4498-87b4-4c3d-bcb9-7634ddfd45cc None admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:02.139166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 640/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Mon May 23 15:20:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:20:11.979708 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=30 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:12.134940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:06.046547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc4b2-f0a4-4e8b-8eb1-9d5363b62bc4 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:06.047134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc4b2-f0a4-4e8b-8eb1-9d5363b62bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:06.047134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc4b2-f0a4-4e8b-8eb1-9d5363b62bc4 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:06.048631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 667/1307] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:06.053817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a86675-591d-4231-a7e0-63b7a5e5be75 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:06.054215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a86675-591d-4231-a7e0-63b7a5e5be75 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:06.054436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a86675-591d-4231-a7e0-63b7a5e5be75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:06.131978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-72a86675-591d-4231-a7e0-63b7a5e5be75 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:06.133217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-72a86675-591d-4231-a7e0-63b7a5e5be75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:06.264921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 641/1308] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:06.302420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9c803-b6bb-4290-9f70-6fa9bf5db5ac None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:06.302738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9c803-b6bb-4290-9f70-6fa9bf5db5ac None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:06.302913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9c803-b6bb-4290-9f70-6fa9bf5db5ac None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:06.304664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 668/1309] 10.222.0.30 () {54 vars in 964 bytes} [Mon May 23 15:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:06.310932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e1f8c-817e-4cb4-a199-99667bbd575f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:06.311224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e1f8c-817e-4cb4-a199-99667bbd575f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:06.311501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e1f8c-817e-4cb4-a199-99667bbd575f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:06.396980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8f0e1f8c-817e-4cb4-a199-99667bbd575f None None] MFA Rules not processed for user `68d85e20dc884cd0bad5bb6c304776ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:06.398727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8f0e1f8c-817e-4cb4-a199-99667bbd575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:06.422213 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:06.489855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 642/1310] 10.222.0.30 () {58 vars in 1061 bytes} [Mon May 23 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:06.500674 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9c803-b6bb-4290-9f70-6fa9bf5db5ac None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:06.502782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-92d9c803-b6bb-4290-9f70-6fa9bf5db5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:06.682077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9c803-b6bb-4290-9f70-6fa9bf5db5ac None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:06.774135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5425fe8-3719-4eb9-b4d7-15a53c999764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:06.774923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5425fe8-3719-4eb9-b4d7-15a53c999764 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:06.775218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5425fe8-3719-4eb9-b4d7-15a53c999764 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:06.775468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5425fe8-3719-4eb9-b4d7-15a53c999764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:06.777151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d5425fe8-3719-4eb9-b4d7-15a53c999764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:06.838573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:06.911413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5425fe8-3719-4eb9-b4d7-15a53c999764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:06.912907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5425fe8-3719-4eb9-b4d7-15a53c999764 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:06.932323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 669/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:09.508342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52884e8c-8cfb-424d-af1b-aa14b208add1 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:09.508899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52884e8c-8cfb-424d-af1b-aa14b208add1 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:09.508899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52884e8c-8cfb-424d-af1b-aa14b208add1 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:09.510316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 643/1312] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:09.516424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9a08d-f6ae-4bb8-b581-8ce7cd3b287b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:09.516978 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9a08d-f6ae-4bb8-b581-8ce7cd3b287b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:09.517074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9a08d-f6ae-4bb8-b581-8ce7cd3b287b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:09.624279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-29f9a08d-f6ae-4bb8-b581-8ce7cd3b287b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:09.626308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-29f9a08d-f6ae-4bb8-b581-8ce7cd3b287b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:09.770077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 670/1313] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:09.810503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52884e8c-8cfb-424d-af1b-aa14b208add1 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:09.812409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-52884e8c-8cfb-424d-af1b-aa14b208add1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:09.925405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52884e8c-8cfb-424d-af1b-aa14b208add1 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:10.000724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:10.003994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f55d02-4eeb-43ea-8fef-0eb07bfd717f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:10.004798 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f55d02-4eeb-43ea-8fef-0eb07bfd717f service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:10.005176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f55d02-4eeb-43ea-8fef-0eb07bfd717f service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:10.005550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f55d02-4eeb-43ea-8fef-0eb07bfd717f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:10.008683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-55f55d02-4eeb-43ea-8fef-0eb07bfd717f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:10.125032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f55d02-4eeb-43ea-8fef-0eb07bfd717f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:10.126518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f55d02-4eeb-43ea-8fef-0eb07bfd717f service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:10.146510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 644/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:12.714274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf79c-6b2d-4b13-9535-a817dce41526 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.714827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf79c-6b2d-4b13-9535-a817dce41526 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.714827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf79c-6b2d-4b13-9535-a817dce41526 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.716219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 671/1315] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:12.721442 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877a8b3-3da1-4986-b6d6-cfe024c150fc None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.721839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877a8b3-3da1-4986-b6d6-cfe024c150fc None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.721989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877a8b3-3da1-4986-b6d6-cfe024c150fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.787168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4877a8b3-3da1-4986-b6d6-cfe024c150fc None None] MFA Rules not processed for user `61613a3a760a44eea15c4d454557a296`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:12.788375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4877a8b3-3da1-4986-b6d6-cfe024c150fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:12.839431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:12.884745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 645/1316] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:12.924832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf79c-6b2d-4b13-9535-a817dce41526 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:12.926416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2dccf79c-6b2d-4b13-9535-a817dce41526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:12.936039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:13.028318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf79c-6b2d-4b13-9535-a817dce41526 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:13.101536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be50fd-9ea4-4123-8ac0-b9b60a0daa1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:13.102204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be50fd-9ea4-4123-8ac0-b9b60a0daa1e service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:13.102484 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be50fd-9ea4-4123-8ac0-b9b60a0daa1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:13.102756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be50fd-9ea4-4123-8ac0-b9b60a0daa1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:13.104395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-30be50fd-9ea4-4123-8ac0-b9b60a0daa1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:13.165338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:13.211656 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be50fd-9ea4-4123-8ac0-b9b60a0daa1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:13.213119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be50fd-9ea4-4123-8ac0-b9b60a0daa1e service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:13.231771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 672/1317] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:16.000595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8abbe9-8996-4324-9fda-e3f70934d61c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:16.001152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8abbe9-8996-4324-9fda-e3f70934d61c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:16.001152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8abbe9-8996-4324-9fda-e3f70934d61c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:16.002485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 646/1318] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:16.007706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033c2e6-505d-4cd8-826e-3a6e833b3945 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:16.008077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033c2e6-505d-4cd8-826e-3a6e833b3945 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:16.008269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033c2e6-505d-4cd8-826e-3a6e833b3945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:16.074203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1033c2e6-505d-4cd8-826e-3a6e833b3945 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:16.075450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1033c2e6-505d-4cd8-826e-3a6e833b3945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:16.173679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 673/1319] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:16.211567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8abbe9-8996-4324-9fda-e3f70934d61c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:16.213148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ac8abbe9-8996-4324-9fda-e3f70934d61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:16.316584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8abbe9-8996-4324-9fda-e3f70934d61c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:16.395149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9838a-1788-45cd-901d-cb7e770cc66f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:16.395802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9838a-1788-45cd-901d-cb7e770cc66f service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:16.396084 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9838a-1788-45cd-901d-cb7e770cc66f service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:16.396358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9838a-1788-45cd-901d-cb7e770cc66f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:16.400524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-24c9838a-1788-45cd-901d-cb7e770cc66f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:16.410784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:16.510006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9838a-1788-45cd-901d-cb7e770cc66f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:16.511454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9838a-1788-45cd-901d-cb7e770cc66f service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:16.528908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 647/1320] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:19.404921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806a702-fe2b-4f35-baa8-8e37f62275c1 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.405511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806a702-fe2b-4f35-baa8-8e37f62275c1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.405511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806a702-fe2b-4f35-baa8-8e37f62275c1 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.407897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 674/1321] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:19.413327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d90010-4705-4f30-acfe-a2a02950ad03 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.413760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d90010-4705-4f30-acfe-a2a02950ad03 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.413887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d90010-4705-4f30-acfe-a2a02950ad03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.478069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-53d90010-4705-4f30-acfe-a2a02950ad03 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:19.479265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-53d90010-4705-4f30-acfe-a2a02950ad03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:19.538274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.577154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 648/1322] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:19.612600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806a702-fe2b-4f35-baa8-8e37f62275c1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:19.614151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b806a702-fe2b-4f35-baa8-8e37f62275c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:19.633779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.719682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806a702-fe2b-4f35-baa8-8e37f62275c1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:19.799247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494b634-b63e-421c-a3c0-d1ac98782662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:19.799898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494b634-b63e-421c-a3c0-d1ac98782662 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.800175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494b634-b63e-421c-a3c0-d1ac98782662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.800441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494b634-b63e-421c-a3c0-d1ac98782662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.802144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f494b634-b63e-421c-a3c0-d1ac98782662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:19.872567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.912686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494b634-b63e-421c-a3c0-d1ac98782662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:19.914190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494b634-b63e-421c-a3c0-d1ac98782662 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:19.933036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 675/1323] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:22.207985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b39c34-34f5-4869-8e56-0a76dedeba23 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.208546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b39c34-34f5-4869-8e56-0a76dedeba23 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.208546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b39c34-34f5-4869-8e56-0a76dedeba23 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.209982 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 649/1324] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:22.215363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec086f-9e16-4593-894e-cdb413d4303a None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.215796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec086f-9e16-4593-894e-cdb413d4303a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.215919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec086f-9e16-4593-894e-cdb413d4303a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.280724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-feec086f-9e16-4593-894e-cdb413d4303a None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:22.282039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-feec086f-9e16-4593-894e-cdb413d4303a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:22.384487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 676/1325] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:22.435651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb51267-a600-42c8-875a-8450de776b39 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.436019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb51267-a600-42c8-875a-8450de776b39 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.436360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb51267-a600-42c8-875a-8450de776b39 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.438001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 650/1326] 10.222.0.30 () {54 vars in 964 bytes} [Mon May 23 15:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:22.444203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec3b0c-52d7-48bf-98d3-2a1903b4943a None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.444487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec3b0c-52d7-48bf-98d3-2a1903b4943a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.444760 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec3b0c-52d7-48bf-98d3-2a1903b4943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.474196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:22.519056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-2bec3b0c-52d7-48bf-98d3-2a1903b4943a None None] MFA Rules not processed for user `68d85e20dc884cd0bad5bb6c304776ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:22.520227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2bec3b0c-52d7-48bf-98d3-2a1903b4943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:22.596944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 677/1327] 10.222.0.30 () {58 vars in 1061 bytes} [Mon May 23 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:21:22.606328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb51267-a600-42c8-875a-8450de776b39 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:22.607845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-2bb51267-a600-42c8-875a-8450de776b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:22.714644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb51267-a600-42c8-875a-8450de776b39 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:22.792933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de9f77-77a7-4bed-b604-a02d30c46152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:22.793591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9f77-77a7-4bed-b604-a02d30c46152 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.793876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9f77-77a7-4bed-b604-a02d30c46152 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.794153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9f77-77a7-4bed-b604-a02d30c46152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.795818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b6de9f77-77a7-4bed-b604-a02d30c46152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:22.815203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:22.907584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de9f77-77a7-4bed-b604-a02d30c46152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:22.909072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de9f77-77a7-4bed-b604-a02d30c46152 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:22.927521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 651/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:25.461057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470977a0-1008-48f6-b4db-69e75ca08d1d None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.461617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470977a0-1008-48f6-b4db-69e75ca08d1d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.461617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470977a0-1008-48f6-b4db-69e75ca08d1d None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.463014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 678/1329] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:25.468366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e286549-2b23-42be-b5e5-794f582dbd57 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.468830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e286549-2b23-42be-b5e5-794f582dbd57 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.469151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e286549-2b23-42be-b5e5-794f582dbd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.533723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-7e286549-2b23-42be-b5e5-794f582dbd57 None None] MFA Rules not processed for user `61613a3a760a44eea15c4d454557a296`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:25.534923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7e286549-2b23-42be-b5e5-794f582dbd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:25.601635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:25.634649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 652/1330] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:25.671443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470977a0-1008-48f6-b4db-69e75ca08d1d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.673205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-470977a0-1008-48f6-b4db-69e75ca08d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:25.779405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470977a0-1008-48f6-b4db-69e75ca08d1d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.803465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:25.857385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10dd72-5bdc-43c9-bb25-6feb0e2e4d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.858039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10dd72-5bdc-43c9-bb25-6feb0e2e4d74 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.858320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10dd72-5bdc-43c9-bb25-6feb0e2e4d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.858595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10dd72-5bdc-43c9-bb25-6feb0e2e4d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.860788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ea10dd72-5bdc-43c9-bb25-6feb0e2e4d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:25.969365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10dd72-5bdc-43c9-bb25-6feb0e2e4d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.970810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10dd72-5bdc-43c9-bb25-6feb0e2e4d74 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.991381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 679/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:21:28.463577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc385b-c07d-4717-a478-515a865263ca None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:28.464138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc385b-c07d-4717-a478-515a865263ca None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:28.464138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc385b-c07d-4717-a478-515a865263ca None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:28.465562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 653/1332] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:28.471419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1124a76-e138-4bf1-b8e1-7af372d21ae5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:28.471877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1124a76-e138-4bf1-b8e1-7af372d21ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:28.471970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1124a76-e138-4bf1-b8e1-7af372d21ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:28.539219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:28.543958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d1124a76-e138-4bf1-b8e1-7af372d21ae5 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:28.545843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d1124a76-e138-4bf1-b8e1-7af372d21ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:28.671090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 680/1333] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:28.707249 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc385b-c07d-4717-a478-515a865263ca None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:28.708836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-59dc385b-c07d-4717-a478-515a865263ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:28.820171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc385b-c07d-4717-a478-515a865263ca None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:28.895291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63326-2876-4721-af7d-06bdcfc94a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:28.895946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63326-2876-4721-af7d-06bdcfc94a2a service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:28.896229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63326-2876-4721-af7d-06bdcfc94a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:28.896503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63326-2876-4721-af7d-06bdcfc94a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:28.898184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1ec63326-2876-4721-af7d-06bdcfc94a2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:28.925447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:29.008305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63326-2876-4721-af7d-06bdcfc94a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:29.009693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63326-2876-4721-af7d-06bdcfc94a2a service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:29.028087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 654/1334] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:31.282178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d6078-d563-47b6-9aaf-42d0f993925a None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:31.282740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d6078-d563-47b6-9aaf-42d0f993925a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:31.282740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d6078-d563-47b6-9aaf-42d0f993925a None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:31.284086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 681/1335] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:31.289248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d5ee0-7004-4ba3-ad89-0b8ea6039907 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:31.289635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d5ee0-7004-4ba3-ad89-0b8ea6039907 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:31.289802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d5ee0-7004-4ba3-ad89-0b8ea6039907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:31.353943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-fc5d5ee0-7004-4ba3-ad89-0b8ea6039907 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:31.355127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-fc5d5ee0-7004-4ba3-ad89-0b8ea6039907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:31.428304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:31.451104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 655/1336] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:21:31.491104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d6078-d563-47b6-9aaf-42d0f993925a None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:31.492652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a67d6078-d563-47b6-9aaf-42d0f993925a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:31.603097 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d6078-d563-47b6-9aaf-42d0f993925a None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:31.631681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:31.675699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4522a48-841f-4d54-8f31-086df932409b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:31.676348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4522a48-841f-4d54-8f31-086df932409b service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:31.676632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4522a48-841f-4d54-8f31-086df932409b service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:31.676939 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4522a48-841f-4d54-8f31-086df932409b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:31.678569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d4522a48-841f-4d54-8f31-086df932409b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:31.784987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4522a48-841f-4d54-8f31-086df932409b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:31.786424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4522a48-841f-4d54-8f31-086df932409b service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:31.804510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 682/1337] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:34.707234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527043b-3e59-4e99-a249-0018aff6533a None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.707754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527043b-3e59-4e99-a249-0018aff6533a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.707754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527043b-3e59-4e99-a249-0018aff6533a None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.709185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 656/1338] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:34.714295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83d0b-07e3-45ee-9cec-3d71f98d681d None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.714681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83d0b-07e3-45ee-9cec-3d71f98d681d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.714866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83d0b-07e3-45ee-9cec-3d71f98d681d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.781758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-70e83d0b-07e3-45ee-9cec-3d71f98d681d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:34.782974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-70e83d0b-07e3-45ee-9cec-3d71f98d681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:34.791256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:34.884699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 683/1339] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:34.921315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527043b-3e59-4e99-a249-0018aff6533a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:34.922939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d527043b-3e59-4e99-a249-0018aff6533a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:35.024796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527043b-3e59-4e99-a249-0018aff6533a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.098237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d6811-cb03-43d0-92b0-d7b9a3c53972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.098882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d6811-cb03-43d0-92b0-d7b9a3c53972 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.099162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d6811-cb03-43d0-92b0-d7b9a3c53972 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.099433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d6811-cb03-43d0-92b0-d7b9a3c53972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.101097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e56d6811-cb03-43d0-92b0-d7b9a3c53972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:35.136201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:35.225593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d6811-cb03-43d0-92b0-d7b9a3c53972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.227057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d6811-cb03-43d0-92b0-d7b9a3c53972 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.244809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 657/1340] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:39.548207 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac331f7-744d-43cb-bd25-5a83be2306e8 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.548784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac331f7-744d-43cb-bd25-5a83be2306e8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.548784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac331f7-744d-43cb-bd25-5a83be2306e8 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.550179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 684/1341] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:39.555344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba4cfb-c926-44c5-8c58-9c4ef38f3434 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.555783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba4cfb-c926-44c5-8c58-9c4ef38f3434 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.555886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba4cfb-c926-44c5-8c58-9c4ef38f3434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.630725 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b0ba4cfb-c926-44c5-8c58-9c4ef38f3434 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:39.631955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b0ba4cfb-c926-44c5-8c58-9c4ef38f3434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:39.724328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:39.737743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 658/1342] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:39.776923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac331f7-744d-43cb-bd25-5a83be2306e8 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.779132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2ac331f7-744d-43cb-bd25-5a83be2306e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:39.884470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac331f7-744d-43cb-bd25-5a83be2306e8 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:39.888025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2b118-3280-44b7-a12d-4d2c8badb7e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:39.888678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2b118-3280-44b7-a12d-4d2c8badb7e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.889004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2b118-3280-44b7-a12d-4d2c8badb7e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.889278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2b118-3280-44b7-a12d-4d2c8badb7e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.905809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2b118-3280-44b7-a12d-4d2c8badb7e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:39.907320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2b118-3280-44b7-a12d-4d2c8badb7e7 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:39.920331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 685/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:42.755292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affa013-f748-4b44-9892-a3a3b041bddd None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.755810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affa013-f748-4b44-9892-a3a3b041bddd None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.755810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affa013-f748-4b44-9892-a3a3b041bddd None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.757259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 659/1344] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:42.762852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63750e75-0560-4171-a8da-898608d6c357 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.763254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63750e75-0560-4171-a8da-898608d6c357 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.763479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63750e75-0560-4171-a8da-898608d6c357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.791926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-63750e75-0560-4171-a8da-898608d6c357 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:42.793143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-63750e75-0560-4171-a8da-898608d6c357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:42.821380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:42.871472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 686/1345] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:42.908914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affa013-f748-4b44-9892-a3a3b041bddd None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.925711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affa013-f748-4b44-9892-a3a3b041bddd None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.937648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afafbf0-8ac6-4085-9540-231f60b32a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.938305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afafbf0-8ac6-4085-9540-231f60b32a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.938605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afafbf0-8ac6-4085-9540-231f60b32a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.938972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afafbf0-8ac6-4085-9540-231f60b32a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.950612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afafbf0-8ac6-4085-9540-231f60b32a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.952061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afafbf0-8ac6-4085-9540-231f60b32a19 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.962320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 660/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:47.084822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fbdd3-4fcf-4ab8-866f-55b9856c8590 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.085329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fbdd3-4fcf-4ab8-866f-55b9856c8590 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.085428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fbdd3-4fcf-4ab8-866f-55b9856c8590 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.087008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 687/1347] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:47.092155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac7f60-08dd-430e-a492-59eee83903e1 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.092570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac7f60-08dd-430e-a492-59eee83903e1 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.092704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac7f60-08dd-430e-a492-59eee83903e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.110412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-7cac7f60-08dd-430e-a492-59eee83903e1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:47.111588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7cac7f60-08dd-430e-a492-59eee83903e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:47.119405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 661/1348] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:47.157318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fbdd3-4fcf-4ab8-866f-55b9856c8590 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.168434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fbdd3-4fcf-4ab8-866f-55b9856c8590 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.172547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fac0d-2e33-4293-b614-5843854db4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.173407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fac0d-2e33-4293-b614-5843854db4bb service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.173595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fac0d-2e33-4293-b614-5843854db4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.173874 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fac0d-2e33-4293-b614-5843854db4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.183491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fac0d-2e33-4293-b614-5843854db4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.184969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fac0d-2e33-4293-b614-5843854db4bb service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.195257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 688/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:21:52.602043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400085-89ab-4dc4-baca-2ea694f0c705 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.602450 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400085-89ab-4dc4-baca-2ea694f0c705 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.602572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400085-89ab-4dc4-baca-2ea694f0c705 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.604229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 662/1350] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:52.610327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb1a7-0f93-4c77-9f8e-bcbcadf9130f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.611047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb1a7-0f93-4c77-9f8e-bcbcadf9130f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.611471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb1a7-0f93-4c77-9f8e-bcbcadf9130f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.695601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0adfb1a7-0f93-4c77-9f8e-bcbcadf9130f None None] MFA Rules not processed for user `3a7dd5a0102d4d3493f09e809cb3caaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:52.697435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0adfb1a7-0f93-4c77-9f8e-bcbcadf9130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:52.837436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 689/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:21:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:57.101609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ba910-789b-4b13-8c7c-71f84c5d3dc4 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.102143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ba910-789b-4b13-8c7c-71f84c5d3dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.102143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ba910-789b-4b13-8c7c-71f84c5d3dc4 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.103796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 663/1352] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:57.105065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=13 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:57.110533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0270c88-e86a-46c0-825f-c6ec76b07f80 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.111007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0270c88-e86a-46c0-825f-c6ec76b07f80 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.111109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0270c88-e86a-46c0-825f-c6ec76b07f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.130013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d0270c88-e86a-46c0-825f-c6ec76b07f80 None None] MFA Rules not processed for user `3a7dd5a0102d4d3493f09e809cb3caaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:57.131213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d0270c88-e86a-46c0-825f-c6ec76b07f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:57.140093 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 690/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:21:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:21:58.701897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40de15-4c7e-4572-bb4f-812e525b3ad5 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.702409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40de15-4c7e-4572-bb4f-812e525b3ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.702497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40de15-4c7e-4572-bb4f-812e525b3ad5 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.704355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 664/1354] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:58.709502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a3fac-d9d1-467b-8d8d-fa46aa4afd3b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.709926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a3fac-d9d1-467b-8d8d-fa46aa4afd3b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.710055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a3fac-d9d1-467b-8d8d-fa46aa4afd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.728748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-cb3a3fac-d9d1-467b-8d8d-fa46aa4afd3b None None] MFA Rules not processed for user `3a7dd5a0102d4d3493f09e809cb3caaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:58.729965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-cb3a3fac-d9d1-467b-8d8d-fa46aa4afd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:58.740933 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 691/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:21:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:02.602777 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353efcd-36a7-43c7-b625-1a112ca26963 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:02.603300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353efcd-36a7-43c7-b625-1a112ca26963 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:02.603300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353efcd-36a7-43c7-b625-1a112ca26963 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:02.608574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 665/1356] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:02.610737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbb101-2688-434a-a9d7-f9bd3ae7841b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:02.611178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbb101-2688-434a-a9d7-f9bd3ae7841b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:02.611284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbb101-2688-434a-a9d7-f9bd3ae7841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:02.632915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4ddbb101-2688-434a-a9d7-f9bd3ae7841b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:02.634389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4ddbb101-2688-434a-a9d7-f9bd3ae7841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:02.642291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 692/1357] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:02.698747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee63215-cf23-4376-9403-9a2135af8cd7 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:02.699110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee63215-cf23-4376-9403-9a2135af8cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:02.699465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee63215-cf23-4376-9403-9a2135af8cd7 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:02.700960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 666/1358] 10.222.0.30 () {54 vars in 961 bytes} [Mon May 23 15:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:02.708818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f4001-daed-4a0a-9de9-73c999e236f0 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:02.709441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f4001-daed-4a0a-9de9-73c999e236f0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:02.710157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f4001-daed-4a0a-9de9-73c999e236f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:02.759418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:02.779512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-128f4001-daed-4a0a-9de9-73c999e236f0 None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:02.783645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-128f4001-daed-4a0a-9de9-73c999e236f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:02.880581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 693/1359] 10.222.0.30 () {58 vars in 1058 bytes} [Mon May 23 15:22:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:02.888495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee63215-cf23-4376-9403-9a2135af8cd7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:02.907014 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee63215-cf23-4376-9403-9a2135af8cd7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:02.912504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7fca6-bc8c-476a-a52c-886b720ef6ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:02.913176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7fca6-bc8c-476a-a52c-886b720ef6ec service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:02.913457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7fca6-bc8c-476a-a52c-886b720ef6ec service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:02.913728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7fca6-bc8c-476a-a52c-886b720ef6ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:02.922620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7fca6-bc8c-476a-a52c-886b720ef6ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:02.924380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7fca6-bc8c-476a-a52c-886b720ef6ec service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:02.935340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 667/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:05.385153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a8b20-1f06-480e-b31c-dd4ae068537f None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.385880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a8b20-1f06-480e-b31c-dd4ae068537f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.385880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a8b20-1f06-480e-b31c-dd4ae068537f None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.387392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 694/1361] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:05.393083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82271d80-254e-416b-b31b-60fff31be824 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.393981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82271d80-254e-416b-b31b-60fff31be824 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.394481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82271d80-254e-416b-b31b-60fff31be824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.418754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-82271d80-254e-416b-b31b-60fff31be824 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:05.421970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-82271d80-254e-416b-b31b-60fff31be824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:05.444306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 668/1362] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:05.483716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a8b20-1f06-480e-b31c-dd4ae068537f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.499267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a8b20-1f06-480e-b31c-dd4ae068537f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.505352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e79fec-bb69-4574-9f5c-fd0996f0edd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.506106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e79fec-bb69-4574-9f5c-fd0996f0edd1 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.506462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e79fec-bb69-4574-9f5c-fd0996f0edd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.506836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e79fec-bb69-4574-9f5c-fd0996f0edd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.516217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e79fec-bb69-4574-9f5c-fd0996f0edd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.517929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e79fec-bb69-4574-9f5c-fd0996f0edd1 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.527937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 695/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:08.286216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b632db-d360-4ca8-85c9-f55ccbe4c041 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.287036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b632db-d360-4ca8-85c9-f55ccbe4c041 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.287471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b632db-d360-4ca8-85c9-f55ccbe4c041 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.289044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 669/1364] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:08.294480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2859b6-70e9-443c-b6fe-1c5d4f163471 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.294918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2859b6-70e9-443c-b6fe-1c5d4f163471 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.295014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2859b6-70e9-443c-b6fe-1c5d4f163471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.319291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-5c2859b6-70e9-443c-b6fe-1c5d4f163471 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:08.320483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5c2859b6-70e9-443c-b6fe-1c5d4f163471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:08.328779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 696/1365] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:08.386181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba264-05b9-403a-a591-a869b2afad79 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.386580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba264-05b9-403a-a591-a869b2afad79 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.386749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba264-05b9-403a-a591-a869b2afad79 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.388408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 670/1366] 10.222.0.30 () {54 vars in 961 bytes} [Mon May 23 15:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:08.394716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d71d7a-31f2-4e51-8de3-b7b8be3a04a5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.395076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d71d7a-31f2-4e51-8de3-b7b8be3a04a5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.395435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d71d7a-31f2-4e51-8de3-b7b8be3a04a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.413603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d5d71d7a-31f2-4e51-8de3-b7b8be3a04a5 None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:08.414788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d5d71d7a-31f2-4e51-8de3-b7b8be3a04a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:08.424494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 697/1367] 10.222.0.30 () {58 vars in 1058 bytes} [Mon May 23 15:22:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:22:08.432328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba264-05b9-403a-a591-a869b2afad79 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:08.447260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba264-05b9-403a-a591-a869b2afad79 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:08.452694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e004b-c7f4-45a2-af72-9ea0bdb75dea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:08.453591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e004b-c7f4-45a2-af72-9ea0bdb75dea service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.453938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e004b-c7f4-45a2-af72-9ea0bdb75dea service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.454284 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e004b-c7f4-45a2-af72-9ea0bdb75dea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.462417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e004b-c7f4-45a2-af72-9ea0bdb75dea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:08.464039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e004b-c7f4-45a2-af72-9ea0bdb75dea service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:08.473606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 671/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:10.795900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb425ee-8646-41f5-92d2-58b58e1a2a62 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:10.796826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb425ee-8646-41f5-92d2-58b58e1a2a62 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:10.797195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb425ee-8646-41f5-92d2-58b58e1a2a62 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:10.798597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 698/1369] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:10.804077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9227-2e14-4c12-a8af-1639a0ac439b None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:10.804565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9227-2e14-4c12-a8af-1639a0ac439b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:10.804654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9227-2e14-4c12-a8af-1639a0ac439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:10.824793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-495f9227-2e14-4c12-a8af-1639a0ac439b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:10.826029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-495f9227-2e14-4c12-a8af-1639a0ac439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:10.834752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 672/1370] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:10.880232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb425ee-8646-41f5-92d2-58b58e1a2a62 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:10.896854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb425ee-8646-41f5-92d2-58b58e1a2a62 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:10.902709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8439fd5-bf31-4b88-a0b6-dbcb80eaeb48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:10.903465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8439fd5-bf31-4b88-a0b6-dbcb80eaeb48 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:10.903845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8439fd5-bf31-4b88-a0b6-dbcb80eaeb48 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:10.904214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8439fd5-bf31-4b88-a0b6-dbcb80eaeb48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:10.914036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8439fd5-bf31-4b88-a0b6-dbcb80eaeb48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:10.915963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8439fd5-bf31-4b88-a0b6-dbcb80eaeb48 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:10.928437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 699/1371] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:11.070451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206b94b-812b-46e5-aa8f-2372e02c7376 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.070773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206b94b-812b-46e5-aa8f-2372e02c7376 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.071054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206b94b-812b-46e5-aa8f-2372e02c7376 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.072567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 673/1372] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:11.079554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c00e0-e074-4c6d-ae3a-4c27e9f0ee1e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.080134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c00e0-e074-4c6d-ae3a-4c27e9f0ee1e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.080585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c00e0-e074-4c6d-ae3a-4c27e9f0ee1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.101258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1a3c00e0-e074-4c6d-ae3a-4c27e9f0ee1e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:11.102859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1a3c00e0-e074-4c6d-ae3a-4c27e9f0ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:11.123639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 700/1373] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:11.447400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea393850-2915-4a41-b546-5311ee54b60b None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.447602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea393850-2915-4a41-b546-5311ee54b60b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.447836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea393850-2915-4a41-b546-5311ee54b60b None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.449512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 674/1374] 10.222.0.30 () {54 vars in 963 bytes} [Mon May 23 15:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:11.456312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73653921-db61-4d66-a7e9-e44af80f473c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.456642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73653921-db61-4d66-a7e9-e44af80f473c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.456911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73653921-db61-4d66-a7e9-e44af80f473c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.508757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-73653921-db61-4d66-a7e9-e44af80f473c None None] MFA Rules not processed for user `36daf80a4cbc4cfc8962c7924433d257`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:11.509977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-73653921-db61-4d66-a7e9-e44af80f473c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:11.572904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 701/1375] 10.222.0.30 () {58 vars in 1060 bytes} [Mon May 23 15:22:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:11.581211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea393850-2915-4a41-b546-5311ee54b60b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.597427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea393850-2915-4a41-b546-5311ee54b60b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:11.601004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd961a-394e-44c5-a3ff-4253aa41401e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:11.601735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd961a-394e-44c5-a3ff-4253aa41401e service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.602085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd961a-394e-44c5-a3ff-4253aa41401e service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.602433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd961a-394e-44c5-a3ff-4253aa41401e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.616601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd961a-394e-44c5-a3ff-4253aa41401e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:11.618170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd961a-394e-44c5-a3ff-4253aa41401e service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:11.628310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 675/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:22:13.223214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731cd6d-aeac-4bcc-8e7b-778db504a0d7 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.223804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731cd6d-aeac-4bcc-8e7b-778db504a0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.223804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731cd6d-aeac-4bcc-8e7b-778db504a0d7 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.225217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 702/1377] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:13.231039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eea759-c4cc-42be-9875-c23af2593590 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.231561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eea759-c4cc-42be-9875-c23af2593590 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.231654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eea759-c4cc-42be-9875-c23af2593590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.251425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-85eea759-c4cc-42be-9875-c23af2593590 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:13.252647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-85eea759-c4cc-42be-9875-c23af2593590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:13.260638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 676/1378] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:13.306883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731cd6d-aeac-4bcc-8e7b-778db504a0d7 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.321813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731cd6d-aeac-4bcc-8e7b-778db504a0d7 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.327185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acf3fc-24a4-47a7-9882-2e6f0170ea1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.327853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acf3fc-24a4-47a7-9882-2e6f0170ea1d service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.328132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acf3fc-24a4-47a7-9882-2e6f0170ea1d service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.329076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acf3fc-24a4-47a7-9882-2e6f0170ea1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.338331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acf3fc-24a4-47a7-9882-2e6f0170ea1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.339964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acf3fc-24a4-47a7-9882-2e6f0170ea1d service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.344700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:13.352043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 703/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:15.640049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510310-fff2-4e52-a958-0d2cb222d21c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:15.640969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510310-fff2-4e52-a958-0d2cb222d21c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:15.641421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510310-fff2-4e52-a958-0d2cb222d21c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:15.645687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 677/1380] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:15.648235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8416756-f07d-4df9-aa0c-6850d20b53d3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:15.648661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8416756-f07d-4df9-aa0c-6850d20b53d3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:15.648796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8416756-f07d-4df9-aa0c-6850d20b53d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:15.667432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f8416756-f07d-4df9-aa0c-6850d20b53d3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:15.668618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f8416756-f07d-4df9-aa0c-6850d20b53d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:15.677207 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 704/1381] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:15.722953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea510310-fff2-4e52-a958-0d2cb222d21c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:15.738265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea510310-fff2-4e52-a958-0d2cb222d21c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:15.743679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8bd5f-476d-485e-9ef3-99c089f93a83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:15.744434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8bd5f-476d-485e-9ef3-99c089f93a83 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:15.744824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8bd5f-476d-485e-9ef3-99c089f93a83 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:15.745238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8bd5f-476d-485e-9ef3-99c089f93a83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:15.754088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8bd5f-476d-485e-9ef3-99c089f93a83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:15.755843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8bd5f-476d-485e-9ef3-99c089f93a83 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:15.766388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 678/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:18.028259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110240e8-7d3d-458d-b3f4-9ffcc9dbe725 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.028831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110240e8-7d3d-458d-b3f4-9ffcc9dbe725 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.028831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110240e8-7d3d-458d-b3f4-9ffcc9dbe725 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.030372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 705/1383] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:18.036206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c7624-eaa7-401e-bf39-3ba77b95b742 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.036642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c7624-eaa7-401e-bf39-3ba77b95b742 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.036760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c7624-eaa7-401e-bf39-3ba77b95b742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.056288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-932c7624-eaa7-401e-bf39-3ba77b95b742 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:18.057510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-932c7624-eaa7-401e-bf39-3ba77b95b742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:18.066294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 679/1384] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:22:18.113392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110240e8-7d3d-458d-b3f4-9ffcc9dbe725 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:18.129033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110240e8-7d3d-458d-b3f4-9ffcc9dbe725 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:18.134617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c8979-f3a4-488b-83f7-0d2b55504689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:18.135261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c8979-f3a4-488b-83f7-0d2b55504689 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.135542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c8979-f3a4-488b-83f7-0d2b55504689 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.135817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c8979-f3a4-488b-83f7-0d2b55504689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.145072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c8979-f3a4-488b-83f7-0d2b55504689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:18.146777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c8979-f3a4-488b-83f7-0d2b55504689 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:18.156923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 706/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:20.445905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5166a8d-ba67-41a1-8ce7-cba16bd2c52b None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.446451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5166a8d-ba67-41a1-8ce7-cba16bd2c52b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.446451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5166a8d-ba67-41a1-8ce7-cba16bd2c52b None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.447873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 680/1386] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:20.453489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f00721-52fb-439c-ad52-a56845310eac None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.454106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f00721-52fb-439c-ad52-a56845310eac None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.454106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f00721-52fb-439c-ad52-a56845310eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.472879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-35f00721-52fb-439c-ad52-a56845310eac None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:20.474023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-35f00721-52fb-439c-ad52-a56845310eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:20.482133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 707/1387] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:20.531189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5166a8d-ba67-41a1-8ce7-cba16bd2c52b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:20.546534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5166a8d-ba67-41a1-8ce7-cba16bd2c52b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:20.553003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d70097-27b8-4e5d-b2fa-66d2a1d2cdfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:20.553687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d70097-27b8-4e5d-b2fa-66d2a1d2cdfa service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.553991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d70097-27b8-4e5d-b2fa-66d2a1d2cdfa service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.554271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d70097-27b8-4e5d-b2fa-66d2a1d2cdfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.563911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d70097-27b8-4e5d-b2fa-66d2a1d2cdfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:20.565672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d70097-27b8-4e5d-b2fa-66d2a1d2cdfa service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:20.576125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 681/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:23.156334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bddbc5-877d-4752-8377-9eee47401ef3 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.157626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bddbc5-877d-4752-8377-9eee47401ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.158350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bddbc5-877d-4752-8377-9eee47401ef3 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.159912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 708/1389] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:23.168221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49417dd-c013-4e64-9a18-2d02442ce8ce None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.168757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49417dd-c013-4e64-9a18-2d02442ce8ce None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.169498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49417dd-c013-4e64-9a18-2d02442ce8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.196348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e49417dd-c013-4e64-9a18-2d02442ce8ce None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:23.197624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e49417dd-c013-4e64-9a18-2d02442ce8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:23.206129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 682/1390] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:23.260094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bddbc5-877d-4752-8377-9eee47401ef3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.275447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bddbc5-877d-4752-8377-9eee47401ef3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.281118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d3778-4909-41cc-a088-abd606aedb83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.282409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d3778-4909-41cc-a088-abd606aedb83 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.282893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d3778-4909-41cc-a088-abd606aedb83 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.283172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d3778-4909-41cc-a088-abd606aedb83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.291346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d3778-4909-41cc-a088-abd606aedb83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.292961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d3778-4909-41cc-a088-abd606aedb83 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.302225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 709/1391] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:25.948476 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51fb80-dc1d-485d-874d-715172b6b2ec None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:25.949072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51fb80-dc1d-485d-874d-715172b6b2ec None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:25.949072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51fb80-dc1d-485d-874d-715172b6b2ec None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:25.950341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 683/1392] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:25.956407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde5a25-9d1c-426e-b2a0-d6b4a35025c9 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:25.956940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde5a25-9d1c-426e-b2a0-d6b4a35025c9 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:25.957055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde5a25-9d1c-426e-b2a0-d6b4a35025c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:25.975695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1cde5a25-9d1c-426e-b2a0-d6b4a35025c9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:25.976911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1cde5a25-9d1c-426e-b2a0-d6b4a35025c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:25.984622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 710/1393] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:26.029247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51fb80-dc1d-485d-874d-715172b6b2ec None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:26.043309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51fb80-dc1d-485d-874d-715172b6b2ec None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:26.048640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9904ce4-3bb4-495b-9760-a57abd8f0676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:26.049299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9904ce4-3bb4-495b-9760-a57abd8f0676 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.049578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9904ce4-3bb4-495b-9760-a57abd8f0676 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.049850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9904ce4-3bb4-495b-9760-a57abd8f0676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.057767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9904ce4-3bb4-495b-9760-a57abd8f0676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:26.059353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9904ce4-3bb4-495b-9760-a57abd8f0676 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:26.069154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 684/1394] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:27.437695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f496c-e63b-479f-9380-c0f48ab3a631 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.438855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f496c-e63b-479f-9380-c0f48ab3a631 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.439148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f496c-e63b-479f-9380-c0f48ab3a631 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.440915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 711/1395] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:27.446202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bad4a4-6250-4005-99de-35082e03fb62 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.446657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bad4a4-6250-4005-99de-35082e03fb62 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.446765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bad4a4-6250-4005-99de-35082e03fb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.452716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=52 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:27.466346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-53bad4a4-6250-4005-99de-35082e03fb62 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:27.467532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-53bad4a4-6250-4005-99de-35082e03fb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:27.475543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 685/1396] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:22:27.530776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc61fe-3f29-47ec-8a36-51c76138a1a2 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.531069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc61fe-3f29-47ec-8a36-51c76138a1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.531353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc61fe-3f29-47ec-8a36-51c76138a1a2 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.533033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 712/1397] 10.222.0.30 () {54 vars in 962 bytes} [Mon May 23 15:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:27.537789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fd315-5189-43d3-b2e8-7efc86f6406a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.538083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fd315-5189-43d3-b2e8-7efc86f6406a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.538366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fd315-5189-43d3-b2e8-7efc86f6406a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.586789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d83fd315-5189-43d3-b2e8-7efc86f6406a None None] MFA Rules not processed for user `fb0e53d4b3774959820ac72824b3d4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:27.588015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d83fd315-5189-43d3-b2e8-7efc86f6406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:27.652222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 686/1398] 10.222.0.30 () {58 vars in 1059 bytes} [Mon May 23 15:22:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:27.659882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc61fe-3f29-47ec-8a36-51c76138a1a2 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:27.670493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc61fe-3f29-47ec-8a36-51c76138a1a2 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:27.674423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250954b3-d8b6-4666-bbcb-a2b48170174c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb0e53d4b3774959820ac72824b3d4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:27.675070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250954b3-d8b6-4666-bbcb-a2b48170174c service glance] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.675353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250954b3-d8b6-4666-bbcb-a2b48170174c service glance] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.675780 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250954b3-d8b6-4666-bbcb-a2b48170174c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.684984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250954b3-d8b6-4666-bbcb-a2b48170174c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:27.686534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250954b3-d8b6-4666-bbcb-a2b48170174c service glance] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:27.696514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 713/1399] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:22:28.245267 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0cda5-c9a8-4c7c-abcd-66c96a529ae3 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.245641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0cda5-c9a8-4c7c-abcd-66c96a529ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.245993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0cda5-c9a8-4c7c-abcd-66c96a529ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.298479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-06e0cda5-c9a8-4c7c-abcd-66c96a529ae3 None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:28.299665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-06e0cda5-c9a8-4c7c-abcd-66c96a529ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:28.325251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b552db-3b47-4121-a11a-26f805e5e7ea None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.325533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b552db-3b47-4121-a11a-26f805e5e7ea None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.326117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b552db-3b47-4121-a11a-26f805e5e7ea None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.327984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 714/1400] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:28.333230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade13a0-8d09-44eb-b768-96f0a87d0843 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.333515 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade13a0-8d09-44eb-b768-96f0a87d0843 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.333791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade13a0-8d09-44eb-b768-96f0a87d0843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.351427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-aade13a0-8d09-44eb-b768-96f0a87d0843 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:28.352603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-aade13a0-8d09-44eb-b768-96f0a87d0843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:28.360839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 715/1401] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:22:28.367853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 687/1402] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:22:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:28.410528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade13a0-8d09-44eb-b768-96f0a87d0843 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:28.418230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355983-0c2e-4ccc-8606-f7abda531912 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.418521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355983-0c2e-4ccc-8606-f7abda531912 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.418805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355983-0c2e-4ccc-8606-f7abda531912 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.420226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 688/1403] 10.222.0.30 () {54 vars in 942 bytes} [Mon May 23 15:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:28.423874 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade13a0-8d09-44eb-b768-96f0a87d0843 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:28.426907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb1c95-6622-4dbc-9f10-4526404afe71 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.427193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb1c95-6622-4dbc-9f10-4526404afe71 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.427494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb1c95-6622-4dbc-9f10-4526404afe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.429643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526f7b2-c099-4a96-96f0-863f92962f9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:28.430289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526f7b2-c099-4a96-96f0-863f92962f9d service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.430569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526f7b2-c099-4a96-96f0-863f92962f9d service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.430846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526f7b2-c099-4a96-96f0-863f92962f9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.439846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526f7b2-c099-4a96-96f0-863f92962f9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:28.441477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526f7b2-c099-4a96-96f0-863f92962f9d service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:28.450991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 716/1404] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:28.478042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e1bb1c95-6622-4dbc-9f10-4526404afe71 None None] MFA Rules not processed for user `ff6c489d950a4370bbf01866d2fc8fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:28.479220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e1bb1c95-6622-4dbc-9f10-4526404afe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:28.493069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:28.557435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 689/1405] 10.222.0.30 () {58 vars in 1039 bytes} [Mon May 23 15:22:28 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:22:28.567949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526f7b2-c099-4a96-96f0-863f92962f9d service nova] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:28.580143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526f7b2-c099-4a96-96f0-863f92962f9d service nova] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:28.585742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39273f-fbc3-4bef-839b-d6c9ed0bdda9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:28.586413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39273f-fbc3-4bef-839b-d6c9ed0bdda9 service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.586691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39273f-fbc3-4bef-839b-d6c9ed0bdda9 service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.586963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39273f-fbc3-4bef-839b-d6c9ed0bdda9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.596111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39273f-fbc3-4bef-839b-d6c9ed0bdda9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:28.597781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39273f-fbc3-4bef-839b-d6c9ed0bdda9 service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:28.607072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 717/1406] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:22:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:22:30.791871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96fc28-b1c8-422c-99e3-d1802b1304bf None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.792415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96fc28-b1c8-422c-99e3-d1802b1304bf None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.792415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96fc28-b1c8-422c-99e3-d1802b1304bf None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.793849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 690/1407] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:30.799589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f200fdf-879c-493e-bcdd-3fa548c5b737 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.800024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f200fdf-879c-493e-bcdd-3fa548c5b737 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.800222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f200fdf-879c-493e-bcdd-3fa548c5b737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.819571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1f200fdf-879c-493e-bcdd-3fa548c5b737 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:30.821111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1f200fdf-879c-493e-bcdd-3fa548c5b737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:30.840893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 718/1408] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:30.873751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96fc28-b1c8-422c-99e3-d1802b1304bf None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:30.888919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96fc28-b1c8-422c-99e3-d1802b1304bf None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:30.895781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17f032-47c6-46c3-8c10-c734e956730d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:30.896430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17f032-47c6-46c3-8c10-c734e956730d service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.896713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17f032-47c6-46c3-8c10-c734e956730d service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.897025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17f032-47c6-46c3-8c10-c734e956730d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.907698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17f032-47c6-46c3-8c10-c734e956730d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:30.914254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17f032-47c6-46c3-8c10-c734e956730d service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:30.927343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 691/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:40.813337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=46 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:40.920731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=30 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:52.046174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5622253-a0e1-4ae8-b02a-b8fae65f0f05 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.046774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5622253-a0e1-4ae8-b02a-b8fae65f0f05 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.046774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5622253-a0e1-4ae8-b02a-b8fae65f0f05 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.048127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 719/1410] 10.222.0.30 () {54 vars in 918 bytes} [Mon May 23 15:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:52.053391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97251f84-943a-4f58-9ada-112fc51660bf None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.053838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97251f84-943a-4f58-9ada-112fc51660bf None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.053951 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97251f84-943a-4f58-9ada-112fc51660bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.073497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-97251f84-943a-4f58-9ada-112fc51660bf None None] MFA Rules not processed for user `3a7dd5a0102d4d3493f09e809cb3caaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:52.074694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-97251f84-943a-4f58-9ada-112fc51660bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:52.083763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 692/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Mon May 23 15:22:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:52.097548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43bd43-a289-48ba-b6ce-1700d0f9e223 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.097834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43bd43-a289-48ba-b6ce-1700d0f9e223 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.098107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43bd43-a289-48ba-b6ce-1700d0f9e223 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.099489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 720/1412] 10.222.0.30 () {54 vars in 918 bytes} [Mon May 23 15:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:52.103846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a176456-aa6f-49b3-b0f1-5a8df08e21b8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.104134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a176456-aa6f-49b3-b0f1-5a8df08e21b8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.104413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a176456-aa6f-49b3-b0f1-5a8df08e21b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.122067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8a176456-aa6f-49b3-b0f1-5a8df08e21b8 None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:52.123242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8a176456-aa6f-49b3-b0f1-5a8df08e21b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:52.133166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 693/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Mon May 23 15:22:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:54.791716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568b700-a811-488b-9f34-93fc6138b013 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:54.792278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568b700-a811-488b-9f34-93fc6138b013 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:54.792278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568b700-a811-488b-9f34-93fc6138b013 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:54.793663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 721/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:54.798887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fb756-66be-4b38-8598-dc6f9c8232ec None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:54.799287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fb756-66be-4b38-8598-dc6f9c8232ec None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:54.799440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fb756-66be-4b38-8598-dc6f9c8232ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:54.816699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-7a7fb756-66be-4b38-8598-dc6f9c8232ec None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:54.817894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7a7fb756-66be-4b38-8598-dc6f9c8232ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:54.825507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 694/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:54.874246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568b700-a811-488b-9f34-93fc6138b013 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:54.888762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568b700-a811-488b-9f34-93fc6138b013 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:54.894671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0e79a-0dd6-46e2-927f-8ae129aa059c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:54.895303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0e79a-0dd6-46e2-927f-8ae129aa059c service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:54.895583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0e79a-0dd6-46e2-927f-8ae129aa059c service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:54.895856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0e79a-0dd6-46e2-927f-8ae129aa059c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:54.903955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0e79a-0dd6-46e2-927f-8ae129aa059c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:54.905573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0e79a-0dd6-46e2-927f-8ae129aa059c service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:54.915514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 722/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:55.875555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2ec5-4b62-4e4c-8cf0-bf12a56f3d9f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:55.876095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2ec5-4b62-4e4c-8cf0-bf12a56f3d9f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:55.876095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2ec5-4b62-4e4c-8cf0-bf12a56f3d9f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:55.877495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 695/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:55.882711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d961e-263e-44b0-9226-1f2d010d8528 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:55.883144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d961e-263e-44b0-9226-1f2d010d8528 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:55.883265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d961e-263e-44b0-9226-1f2d010d8528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:55.905221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-da0d961e-263e-44b0-9226-1f2d010d8528 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:55.906416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-da0d961e-263e-44b0-9226-1f2d010d8528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:55.914929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 723/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:56.074198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b2ec5-4b62-4e4c-8cf0-bf12a56f3d9f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:56.088206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b2ec5-4b62-4e4c-8cf0-bf12a56f3d9f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:56.093520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649adf9f-e7db-4c6f-a49d-1d1817bf456b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:56.094149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649adf9f-e7db-4c6f-a49d-1d1817bf456b service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:56.094428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649adf9f-e7db-4c6f-a49d-1d1817bf456b service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:56.094710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649adf9f-e7db-4c6f-a49d-1d1817bf456b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:56.103340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649adf9f-e7db-4c6f-a49d-1d1817bf456b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:56.104978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649adf9f-e7db-4c6f-a49d-1d1817bf456b service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:56.114924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 696/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:57.041401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522647f-a238-410b-9b5f-cb43a215797a None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:57.041971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522647f-a238-410b-9b5f-cb43a215797a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:57.041971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522647f-a238-410b-9b5f-cb43a215797a None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:57.043353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 724/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:57.048788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3b351-850e-40fe-912b-cf2ae1652064 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:57.049316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3b351-850e-40fe-912b-cf2ae1652064 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:57.049413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3b351-850e-40fe-912b-cf2ae1652064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:57.067266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-66c3b351-850e-40fe-912b-cf2ae1652064 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:57.068441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-66c3b351-850e-40fe-912b-cf2ae1652064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:57.076075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 697/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:57.237747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522647f-a238-410b-9b5f-cb43a215797a None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:57.252603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522647f-a238-410b-9b5f-cb43a215797a None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:57.258259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7da487-9b59-4ef3-96da-2f5c0bf6d5ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:57.258897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7da487-9b59-4ef3-96da-2f5c0bf6d5ca service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:57.259182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7da487-9b59-4ef3-96da-2f5c0bf6d5ca service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:57.259463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7da487-9b59-4ef3-96da-2f5c0bf6d5ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:57.267735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7da487-9b59-4ef3-96da-2f5c0bf6d5ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:57.269353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7da487-9b59-4ef3-96da-2f5c0bf6d5ca service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:57.279007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 725/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:58.188157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89674e60-9f04-4f3f-865e-f1757ea67d0f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:58.188702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89674e60-9f04-4f3f-865e-f1757ea67d0f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:58.188702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89674e60-9f04-4f3f-865e-f1757ea67d0f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:58.190134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 698/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:58.195328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dde61f-79d1-4b24-832a-bea2dc21ecb4 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:58.195732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dde61f-79d1-4b24-832a-bea2dc21ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:58.195914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dde61f-79d1-4b24-832a-bea2dc21ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:58.213965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-78dde61f-79d1-4b24-832a-bea2dc21ecb4 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:58.215152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-78dde61f-79d1-4b24-832a-bea2dc21ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:58.223767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 726/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:58.375286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89674e60-9f04-4f3f-865e-f1757ea67d0f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:58.390546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89674e60-9f04-4f3f-865e-f1757ea67d0f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:58.396741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08252eb9-adee-4fcf-a3ef-242da9b87b70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:58.397414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08252eb9-adee-4fcf-a3ef-242da9b87b70 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:58.397694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08252eb9-adee-4fcf-a3ef-242da9b87b70 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:58.397967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08252eb9-adee-4fcf-a3ef-242da9b87b70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:58.406282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08252eb9-adee-4fcf-a3ef-242da9b87b70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:58.407980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08252eb9-adee-4fcf-a3ef-242da9b87b70 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:58.417359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 699/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:22:59.339447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5456f7-45cf-4b3c-b8ff-16921422dd1e None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:59.339992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5456f7-45cf-4b3c-b8ff-16921422dd1e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:59.339992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5456f7-45cf-4b3c-b8ff-16921422dd1e None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:59.341443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 727/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:59.347216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff53a5f-fbd1-4fa5-af47-f2ef14957baf None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:59.347685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff53a5f-fbd1-4fa5-af47-f2ef14957baf None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:59.347828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff53a5f-fbd1-4fa5-af47-f2ef14957baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:59.365691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-3ff53a5f-fbd1-4fa5-af47-f2ef14957baf None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:59.366953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3ff53a5f-fbd1-4fa5-af47-f2ef14957baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:59.375234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 700/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:22:59.524886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5456f7-45cf-4b3c-b8ff-16921422dd1e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:59.539726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5456f7-45cf-4b3c-b8ff-16921422dd1e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:59.545377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98568b-4cdb-4460-ab83-978f4104fc22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:59.546021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98568b-4cdb-4460-ab83-978f4104fc22 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:59.546311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98568b-4cdb-4460-ab83-978f4104fc22 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:59.546585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98568b-4cdb-4460-ab83-978f4104fc22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:59.555635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98568b-4cdb-4460-ab83-978f4104fc22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:59.557259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98568b-4cdb-4460-ab83-978f4104fc22 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:59.567110 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 728/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:00.429559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ea2f9-842b-4ea0-a3c0-b7e880e8e54f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:00.430010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ea2f9-842b-4ea0-a3c0-b7e880e8e54f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:00.430105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ea2f9-842b-4ea0-a3c0-b7e880e8e54f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:00.431498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 701/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:00.436439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5119af0-1fe6-4034-a2bd-afa7ecb455c2 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:00.436835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5119af0-1fe6-4034-a2bd-afa7ecb455c2 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:00.437054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5119af0-1fe6-4034-a2bd-afa7ecb455c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:00.454527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f5119af0-1fe6-4034-a2bd-afa7ecb455c2 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:00.455723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f5119af0-1fe6-4034-a2bd-afa7ecb455c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:00.463393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 729/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:00.616094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ea2f9-842b-4ea0-a3c0-b7e880e8e54f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:00.630837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ea2f9-842b-4ea0-a3c0-b7e880e8e54f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:00.636105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739a0fa-c1dd-4d82-a9c7-05f69dda5b50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:00.636768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739a0fa-c1dd-4d82-a9c7-05f69dda5b50 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:00.637074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739a0fa-c1dd-4d82-a9c7-05f69dda5b50 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:00.637350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739a0fa-c1dd-4d82-a9c7-05f69dda5b50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:00.645502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739a0fa-c1dd-4d82-a9c7-05f69dda5b50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:00.647069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739a0fa-c1dd-4d82-a9c7-05f69dda5b50 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:00.656550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 702/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:01.525098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806ced2-263d-4857-ac4b-12551b6084e5 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:01.525651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806ced2-263d-4857-ac4b-12551b6084e5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:01.525651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806ced2-263d-4857-ac4b-12551b6084e5 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:01.527037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 730/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:01.532255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fadccc-6b6f-4e0f-861e-3faed877d977 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:01.532760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fadccc-6b6f-4e0f-861e-3faed877d977 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:01.533222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fadccc-6b6f-4e0f-861e-3faed877d977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:01.552553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-c7fadccc-6b6f-4e0f-861e-3faed877d977 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:01.553797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c7fadccc-6b6f-4e0f-861e-3faed877d977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:01.562236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 703/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:01.721995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806ced2-263d-4857-ac4b-12551b6084e5 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:01.741070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806ced2-263d-4857-ac4b-12551b6084e5 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:01.749302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85d6be-024c-4472-8c4f-09c9d34f9da6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:01.749960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85d6be-024c-4472-8c4f-09c9d34f9da6 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:01.750239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85d6be-024c-4472-8c4f-09c9d34f9da6 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:01.750511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85d6be-024c-4472-8c4f-09c9d34f9da6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:01.761201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85d6be-024c-4472-8c4f-09c9d34f9da6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:01.763976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85d6be-024c-4472-8c4f-09c9d34f9da6 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:01.777944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 731/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:23:02.632414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b11c1f-e070-433b-9b64-abcacc2720aa None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:02.632941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b11c1f-e070-433b-9b64-abcacc2720aa None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:02.633062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b11c1f-e070-433b-9b64-abcacc2720aa None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:02.634430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 704/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:02.639289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c32d5-a158-49c2-bdff-1645cbe00aac None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:02.639776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c32d5-a158-49c2-bdff-1645cbe00aac None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:02.639895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c32d5-a158-49c2-bdff-1645cbe00aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:02.657529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4f9c32d5-a158-49c2-bdff-1645cbe00aac None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:02.659425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4f9c32d5-a158-49c2-bdff-1645cbe00aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:02.667161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 732/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:02.817974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b11c1f-e070-433b-9b64-abcacc2720aa None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:02.832285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b11c1f-e070-433b-9b64-abcacc2720aa None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:02.837646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c417acc-6c5c-4396-bb01-bd84d4e23a03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:02.838277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c417acc-6c5c-4396-bb01-bd84d4e23a03 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:02.838560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c417acc-6c5c-4396-bb01-bd84d4e23a03 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:02.838838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c417acc-6c5c-4396-bb01-bd84d4e23a03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:02.847374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c417acc-6c5c-4396-bb01-bd84d4e23a03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:02.848978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c417acc-6c5c-4396-bb01-bd84d4e23a03 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:02.858396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 705/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:03.733998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3332d-79d5-498a-824b-d12470e96a01 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:03.734572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3332d-79d5-498a-824b-d12470e96a01 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:03.734572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3332d-79d5-498a-824b-d12470e96a01 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:03.736470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 733/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:03.741657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7cff3-a335-457a-9f56-a606fa696bd3 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:03.742095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7cff3-a335-457a-9f56-a606fa696bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:03.742205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7cff3-a335-457a-9f56-a606fa696bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:03.760738 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-3bf7cff3-a335-457a-9f56-a606fa696bd3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:03.761959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3bf7cff3-a335-457a-9f56-a606fa696bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:03.769800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 706/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:03.920187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3332d-79d5-498a-824b-d12470e96a01 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:03.934259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3332d-79d5-498a-824b-d12470e96a01 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:03.940005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675741f2-65da-4eec-ac7a-14f517a1f647 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:03.940651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675741f2-65da-4eec-ac7a-14f517a1f647 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:03.940979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675741f2-65da-4eec-ac7a-14f517a1f647 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:03.941262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675741f2-65da-4eec-ac7a-14f517a1f647 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:03.949664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675741f2-65da-4eec-ac7a-14f517a1f647 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:03.951240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675741f2-65da-4eec-ac7a-14f517a1f647 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:03.961457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 734/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:06.081037 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68775e20-9313-4925-851c-dea817c4be58 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:06.081652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68775e20-9313-4925-851c-dea817c4be58 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:06.081652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68775e20-9313-4925-851c-dea817c4be58 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:06.082985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 707/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:06.088418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a631e-eba5-495c-a4b6-2fce631f039f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:06.088852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a631e-eba5-495c-a4b6-2fce631f039f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:06.088992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a631e-eba5-495c-a4b6-2fce631f039f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:06.107560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b16a631e-eba5-495c-a4b6-2fce631f039f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:06.108809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b16a631e-eba5-495c-a4b6-2fce631f039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:06.117044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 735/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:23:06.154345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68775e20-9313-4925-851c-dea817c4be58 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:06.165772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68775e20-9313-4925-851c-dea817c4be58 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:06.169305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa13c57-7760-428d-8312-c4f33ed2b808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:06.169938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa13c57-7760-428d-8312-c4f33ed2b808 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:06.170222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa13c57-7760-428d-8312-c4f33ed2b808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:06.170497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa13c57-7760-428d-8312-c4f33ed2b808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:06.179476 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa13c57-7760-428d-8312-c4f33ed2b808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:06.180959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa13c57-7760-428d-8312-c4f33ed2b808 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:06.190413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 708/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:08.598842 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce9b5-7a35-4606-a594-d661ad9cf968 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:08.599362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce9b5-7a35-4606-a594-d661ad9cf968 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:08.599362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce9b5-7a35-4606-a594-d661ad9cf968 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:08.600691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 736/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:08.605973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f99c29-0d43-439f-9a9e-242c773dbfd9 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:08.606460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f99c29-0d43-439f-9a9e-242c773dbfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:08.606649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f99c29-0d43-439f-9a9e-242c773dbfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:08.625151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-39f99c29-0d43-439f-9a9e-242c773dbfd9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:08.626354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-39f99c29-0d43-439f-9a9e-242c773dbfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:08.636133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 709/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:23:08.677838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ce9b5-7a35-4606-a594-d661ad9cf968 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:08.688450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ce9b5-7a35-4606-a594-d661ad9cf968 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:08.692232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f77b3d-e76e-47d7-a1c6-e7a27f817019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:08.692895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f77b3d-e76e-47d7-a1c6-e7a27f817019 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:08.693174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f77b3d-e76e-47d7-a1c6-e7a27f817019 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:08.693449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f77b3d-e76e-47d7-a1c6-e7a27f817019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:08.702400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f77b3d-e76e-47d7-a1c6-e7a27f817019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:08.703946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f77b3d-e76e-47d7-a1c6-e7a27f817019 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:08.713280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 737/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:11.296696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b03c8-aa1f-418b-ba75-164aff27f6d4 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:11.297282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b03c8-aa1f-418b-ba75-164aff27f6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:11.297282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b03c8-aa1f-418b-ba75-164aff27f6d4 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:11.299723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 710/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:11.304772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd38d-1871-440c-8662-523feb12efa0 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:11.305277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd38d-1871-440c-8662-523feb12efa0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:11.305380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd38d-1871-440c-8662-523feb12efa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:11.323401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-7e4dd38d-1871-440c-8662-523feb12efa0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:11.324579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7e4dd38d-1871-440c-8662-523feb12efa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:11.334021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 738/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:11.375087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b03c8-aa1f-418b-ba75-164aff27f6d4 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:11.386612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b03c8-aa1f-418b-ba75-164aff27f6d4 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:11.390106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9079d-e32c-4705-8f5a-4bffcae1dab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:11.390739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9079d-e32c-4705-8f5a-4bffcae1dab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:11.391018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9079d-e32c-4705-8f5a-4bffcae1dab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:11.391291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9079d-e32c-4705-8f5a-4bffcae1dab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:11.400157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9079d-e32c-4705-8f5a-4bffcae1dab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:11.401636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9079d-e32c-4705-8f5a-4bffcae1dab3 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:11.410998 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 711/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:13.755977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a636b0-caba-46a4-8897-597fd11a125e None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:13.756596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a636b0-caba-46a4-8897-597fd11a125e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:13.756596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a636b0-caba-46a4-8897-597fd11a125e None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:13.757940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 739/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:13.763153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47e67b-6246-4913-a6a5-208d5c78f244 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:13.763615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47e67b-6246-4913-a6a5-208d5c78f244 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:13.763721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47e67b-6246-4913-a6a5-208d5c78f244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:13.781343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-fe47e67b-6246-4913-a6a5-208d5c78f244 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:13.782539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-fe47e67b-6246-4913-a6a5-208d5c78f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:13.790204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 712/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:13.829062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a636b0-caba-46a4-8897-597fd11a125e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:13.839665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a636b0-caba-46a4-8897-597fd11a125e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:13.843144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd963c3-0d4c-4d3e-b9f7-37a736470f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:13.843781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd963c3-0d4c-4d3e-b9f7-37a736470f5f service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:13.844064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd963c3-0d4c-4d3e-b9f7-37a736470f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:13.844339 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd963c3-0d4c-4d3e-b9f7-37a736470f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:13.853718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd963c3-0d4c-4d3e-b9f7-37a736470f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:13.855139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd963c3-0d4c-4d3e-b9f7-37a736470f5f service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:13.865228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 740/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:16.163010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b2129-b636-4227-b39d-b10e6137dc70 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:16.163570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b2129-b636-4227-b39d-b10e6137dc70 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:16.163570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b2129-b636-4227-b39d-b10e6137dc70 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:16.164978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 713/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:16.170196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bcd56-e0f5-4c87-8620-e3a3cdfdd067 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:16.170577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bcd56-e0f5-4c87-8620-e3a3cdfdd067 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:16.170753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bcd56-e0f5-4c87-8620-e3a3cdfdd067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:16.188246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1a4bcd56-e0f5-4c87-8620-e3a3cdfdd067 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:16.190003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1a4bcd56-e0f5-4c87-8620-e3a3cdfdd067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:16.198051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 741/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:16.236030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b2129-b636-4227-b39d-b10e6137dc70 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:16.246813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b2129-b636-4227-b39d-b10e6137dc70 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:16.250328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a35f3-19a6-4453-a5b6-eccbaa20cdb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:16.250959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a35f3-19a6-4453-a5b6-eccbaa20cdb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:16.251244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a35f3-19a6-4453-a5b6-eccbaa20cdb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:16.251540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a35f3-19a6-4453-a5b6-eccbaa20cdb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:16.261285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a35f3-19a6-4453-a5b6-eccbaa20cdb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:16.262738 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a35f3-19a6-4453-a5b6-eccbaa20cdb3 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:16.272731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 714/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:18.500102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da9a78-4e71-4db2-813e-eac21594aeec None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:18.500644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da9a78-4e71-4db2-813e-eac21594aeec None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:18.500644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da9a78-4e71-4db2-813e-eac21594aeec None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:18.502096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 742/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:18.507735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9e252-8a7c-4442-938a-eb0c03b68cd3 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:18.508137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9e252-8a7c-4442-938a-eb0c03b68cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:18.508302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9e252-8a7c-4442-938a-eb0c03b68cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:18.525740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-14d9e252-8a7c-4442-938a-eb0c03b68cd3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:18.526928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-14d9e252-8a7c-4442-938a-eb0c03b68cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:18.535458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 715/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:18.572706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da9a78-4e71-4db2-813e-eac21594aeec None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:18.583276 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da9a78-4e71-4db2-813e-eac21594aeec None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:18.586779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0be91-780b-4d61-93c9-921c12e6849d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:18.587416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0be91-780b-4d61-93c9-921c12e6849d service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:18.587700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0be91-780b-4d61-93c9-921c12e6849d service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:18.587975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0be91-780b-4d61-93c9-921c12e6849d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:18.597883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0be91-780b-4d61-93c9-921c12e6849d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:18.599338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0be91-780b-4d61-93c9-921c12e6849d service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:18.608826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 743/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:20.882215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810dc6a-3f4b-4b23-9bcc-d61b7a0dab5e None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:20.882751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810dc6a-3f4b-4b23-9bcc-d61b7a0dab5e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:20.882751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810dc6a-3f4b-4b23-9bcc-d61b7a0dab5e None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:20.884145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 716/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:20.889605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b431-2157-4562-9f4a-abe405a87fb1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:20.890013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b431-2157-4562-9f4a-abe405a87fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:20.890171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b431-2157-4562-9f4a-abe405a87fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:20.896342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=70 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:20.909234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-8272b431-2157-4562-9f4a-abe405a87fb1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:20.910537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8272b431-2157-4562-9f4a-abe405a87fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:20.918315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 744/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:20.956412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810dc6a-3f4b-4b23-9bcc-d61b7a0dab5e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:20.961759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=73 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:20.968376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810dc6a-3f4b-4b23-9bcc-d61b7a0dab5e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:20.972842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83115290-1785-4d30-ab0a-ca37358de570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:20.973507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83115290-1785-4d30-ab0a-ca37358de570 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:20.973787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83115290-1785-4d30-ab0a-ca37358de570 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:20.974058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83115290-1785-4d30-ab0a-ca37358de570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:20.982905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83115290-1785-4d30-ab0a-ca37358de570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:20.984367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83115290-1785-4d30-ab0a-ca37358de570 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:20.995232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 717/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:23.250834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40ac40-8a7a-4d88-a97b-36e62dfe85c1 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:23.251404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40ac40-8a7a-4d88-a97b-36e62dfe85c1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:23.251404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40ac40-8a7a-4d88-a97b-36e62dfe85c1 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:23.252941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 745/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:23.258362 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91b638-24a2-4c3d-8c18-db85595c13b0 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:23.258788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91b638-24a2-4c3d-8c18-db85595c13b0 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:23.258980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91b638-24a2-4c3d-8c18-db85595c13b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:23.276977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-bc91b638-24a2-4c3d-8c18-db85595c13b0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:23.278168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-bc91b638-24a2-4c3d-8c18-db85595c13b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:23.286471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 718/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:23:23.322453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40ac40-8a7a-4d88-a97b-36e62dfe85c1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:23.333720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40ac40-8a7a-4d88-a97b-36e62dfe85c1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:23.337243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f3b41-c02f-4d93-9e84-e22930404ce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:23.337882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f3b41-c02f-4d93-9e84-e22930404ce8 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:23.338168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f3b41-c02f-4d93-9e84-e22930404ce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:23.338444 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f3b41-c02f-4d93-9e84-e22930404ce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:23.347981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f3b41-c02f-4d93-9e84-e22930404ce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:23.349452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f3b41-c02f-4d93-9e84-e22930404ce8 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:23.358715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 746/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:25.603013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551be2b-e940-426b-9213-835afeffe602 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:25.603573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551be2b-e940-426b-9213-835afeffe602 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:25.603573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551be2b-e940-426b-9213-835afeffe602 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:25.605136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 719/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:25.610837 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d348381-5aee-425d-96fc-926e46e74b9b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:25.611227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d348381-5aee-425d-96fc-926e46e74b9b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:25.611394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d348381-5aee-425d-96fc-926e46e74b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:25.628728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-2d348381-5aee-425d-96fc-926e46e74b9b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:25.629941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2d348381-5aee-425d-96fc-926e46e74b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:25.637987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 747/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:25.674776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551be2b-e940-426b-9213-835afeffe602 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:25.695270 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551be2b-e940-426b-9213-835afeffe602 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:25.698790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fe2c7-1378-4dc1-8e49-3db6972cc0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:25.699427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fe2c7-1378-4dc1-8e49-3db6972cc0ce service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:25.699705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fe2c7-1378-4dc1-8e49-3db6972cc0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:25.699977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fe2c7-1378-4dc1-8e49-3db6972cc0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:25.710296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fe2c7-1378-4dc1-8e49-3db6972cc0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:25.711976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fe2c7-1378-4dc1-8e49-3db6972cc0ce service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:25.722436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 720/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:27.927838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26773c24-26fe-41ed-b9da-450407362dfe None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:27.928365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26773c24-26fe-41ed-b9da-450407362dfe None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:27.928365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26773c24-26fe-41ed-b9da-450407362dfe None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:27.929757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 748/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:27.935339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e31cd0-ce18-41a7-9071-66e37b5b6437 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:27.935791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e31cd0-ce18-41a7-9071-66e37b5b6437 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:27.935887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e31cd0-ce18-41a7-9071-66e37b5b6437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:27.953355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e3e31cd0-ce18-41a7-9071-66e37b5b6437 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:27.954534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e3e31cd0-ce18-41a7-9071-66e37b5b6437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:27.963013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 721/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:23:27.999771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26773c24-26fe-41ed-b9da-450407362dfe None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:28.010866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26773c24-26fe-41ed-b9da-450407362dfe None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:28.014348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd86900-2b04-49e2-b4a0-b3b84ffb3f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:28.014985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86900-2b04-49e2-b4a0-b3b84ffb3f56 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:28.015272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86900-2b04-49e2-b4a0-b3b84ffb3f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:28.015551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86900-2b04-49e2-b4a0-b3b84ffb3f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:28.027021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd86900-2b04-49e2-b4a0-b3b84ffb3f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:28.029253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd86900-2b04-49e2-b4a0-b3b84ffb3f56 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:28.043496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 749/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:23:37.948346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=17 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:38.038971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=17 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:24:01.232462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036a4ab-8f59-4cea-a2b8-d9a5fb0d4cc2 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:01.233120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036a4ab-8f59-4cea-a2b8-d9a5fb0d4cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:01.233120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036a4ab-8f59-4cea-a2b8-d9a5fb0d4cc2 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:01.234440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 722/1471] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:24:01.239774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f5799-4ad0-46e2-912f-40fd9f1e4c33 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:01.240223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f5799-4ad0-46e2-912f-40fd9f1e4c33 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:01.240328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f5799-4ad0-46e2-912f-40fd9f1e4c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:01.259301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-048f5799-4ad0-46e2-912f-40fd9f1e4c33 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:24:01.260508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-048f5799-4ad0-46e2-912f-40fd9f1e4c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:01.268353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 750/1472] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:24:01.315176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95493e0c-c2cf-4a97-bac6-82b77a3f2cc5 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:01.315462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95493e0c-c2cf-4a97-bac6-82b77a3f2cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:01.315740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95493e0c-c2cf-4a97-bac6-82b77a3f2cc5 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:01.317143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 723/1473] 10.222.0.30 () {54 vars in 962 bytes} [Mon May 23 15:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:24:01.322534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95219f24-dbdc-4cd3-a6cf-1af9a263de19 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:01.322826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95219f24-dbdc-4cd3-a6cf-1af9a263de19 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:01.323108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95219f24-dbdc-4cd3-a6cf-1af9a263de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:01.341184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-95219f24-dbdc-4cd3-a6cf-1af9a263de19 None None] MFA Rules not processed for user `fb0e53d4b3774959820ac72824b3d4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:24:01.342370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-95219f24-dbdc-4cd3-a6cf-1af9a263de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:01.348827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 751/1474] 10.222.0.30 () {58 vars in 1059 bytes} [Mon May 23 15:24:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:24:01.356169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95493e0c-c2cf-4a97-bac6-82b77a3f2cc5 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:01.367222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95493e0c-c2cf-4a97-bac6-82b77a3f2cc5 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:01.370917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30079052-e3ba-4092-a8ca-572e308337a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb0e53d4b3774959820ac72824b3d4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:01.371614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079052-e3ba-4092-a8ca-572e308337a5 service glance] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:01.371941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079052-e3ba-4092-a8ca-572e308337a5 service glance] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:01.372282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079052-e3ba-4092-a8ca-572e308337a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:01.382595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30079052-e3ba-4092-a8ca-572e308337a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:01.384361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30079052-e3ba-4092-a8ca-572e308337a5 service glance] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:01.394293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 724/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:24:03.747955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb49f6c-f324-45a8-a9a2-f53883346676 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:03.748513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb49f6c-f324-45a8-a9a2-f53883346676 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:03.748513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb49f6c-f324-45a8-a9a2-f53883346676 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:03.749979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 752/1476] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:24:03.756783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463e433-4628-49fc-9692-bf5ac3b0573d None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:03.757368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463e433-4628-49fc-9692-bf5ac3b0573d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:03.757850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463e433-4628-49fc-9692-bf5ac3b0573d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:03.780206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b463e433-4628-49fc-9692-bf5ac3b0573d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:24:03.782098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b463e433-4628-49fc-9692-bf5ac3b0573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:03.791732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 725/1477] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:24:03.818189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb49f6c-f324-45a8-a9a2-f53883346676 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:03.830747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb49f6c-f324-45a8-a9a2-f53883346676 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:03.835919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3630a1e-e499-4d58-99a6-3c87892fd90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:03.836579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3630a1e-e499-4d58-99a6-3c87892fd90c admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:03.836889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3630a1e-e499-4d58-99a6-3c87892fd90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:03.837167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3630a1e-e499-4d58-99a6-3c87892fd90c admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:03.837900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3630a1e-e499-4d58-99a6-3c87892fd90c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:03.839662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING py.warnings [None req-c3630a1e-e499-4d58-99a6-3c87892fd90c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:24:03.839662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: warnings.warn(msg) May 23 15:24:03.839662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: May 23 15:24:03.840815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3630a1e-e499-4d58-99a6-3c87892fd90c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:03.860248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 753/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Mon May 23 15:24:03 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:24:06.112678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc63e-88fc-440b-a49d-4ff38a7c18ef None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:06.113277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc63e-88fc-440b-a49d-4ff38a7c18ef None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:06.113277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc63e-88fc-440b-a49d-4ff38a7c18ef None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:06.115237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 726/1479] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:24:06.120648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaf94d-d4a1-424c-b370-26a064adb145 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:06.121098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaf94d-d4a1-424c-b370-26a064adb145 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:06.121247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaf94d-d4a1-424c-b370-26a064adb145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:06.140041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6deaf94d-d4a1-424c-b370-26a064adb145 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:24:06.141290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6deaf94d-d4a1-424c-b370-26a064adb145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:06.149075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 754/1480] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:24:06.193410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fc63e-88fc-440b-a49d-4ff38a7c18ef None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:06.207505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fc63e-88fc-440b-a49d-4ff38a7c18ef None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:06.212799 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10469ea-3864-44d6-9b83-01770f440731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:06.213461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10469ea-3864-44d6-9b83-01770f440731 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:06.213743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10469ea-3864-44d6-9b83-01770f440731 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:06.214018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10469ea-3864-44d6-9b83-01770f440731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:06.221981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10469ea-3864-44d6-9b83-01770f440731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:06.223574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10469ea-3864-44d6-9b83-01770f440731 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:06.232815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 727/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:24:08.686019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf652c5-4d0a-42d5-b2ab-7f8b618e0c82 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:08.686812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf652c5-4d0a-42d5-b2ab-7f8b618e0c82 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:08.686812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf652c5-4d0a-42d5-b2ab-7f8b618e0c82 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:08.687857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 755/1482] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:24:08.693241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bedec2-f954-41b4-9bea-8e29fd5c9952 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:08.693722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bedec2-f954-41b4-9bea-8e29fd5c9952 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:08.693857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bedec2-f954-41b4-9bea-8e29fd5c9952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:08.711713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-f8bedec2-f954-41b4-9bea-8e29fd5c9952 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:24:08.712935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f8bedec2-f954-41b4-9bea-8e29fd5c9952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:08.720644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 728/1483] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:24:08.756855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf652c5-4d0a-42d5-b2ab-7f8b618e0c82 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:08.767642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf652c5-4d0a-42d5-b2ab-7f8b618e0c82 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:08.771256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d6fac-7f5d-45f0-9e96-9f7cc1de01fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:08.771902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6fac-7f5d-45f0-9e96-9f7cc1de01fe service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:08.772192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6fac-7f5d-45f0-9e96-9f7cc1de01fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:08.772468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6fac-7f5d-45f0-9e96-9f7cc1de01fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:08.781615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d6fac-7f5d-45f0-9e96-9f7cc1de01fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:08.783033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d6fac-7f5d-45f0-9e96-9f7cc1de01fe service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:08.792741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 756/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:24:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:24:11.096183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1aed-c312-4c46-a1ba-228b4f896393 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:11.096721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1aed-c312-4c46-a1ba-228b4f896393 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:11.096721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1aed-c312-4c46-a1ba-228b4f896393 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:11.098176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 729/1485] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:24:11.103544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3741-4e87-4236-9822-49d8c0e9258d None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:11.103983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3741-4e87-4236-9822-49d8c0e9258d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:11.104081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3741-4e87-4236-9822-49d8c0e9258d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:11.121621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-948a3741-4e87-4236-9822-49d8c0e9258d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:24:11.122812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-948a3741-4e87-4236-9822-49d8c0e9258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:11.131167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 757/1486] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:24:11.168680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e1aed-c312-4c46-a1ba-228b4f896393 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:11.179617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e1aed-c312-4c46-a1ba-228b4f896393 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:11.183029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f043a-41fd-498e-9e68-58916cb67f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:11.183652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f043a-41fd-498e-9e68-58916cb67f8d service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:11.183928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f043a-41fd-498e-9e68-58916cb67f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:11.184199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f043a-41fd-498e-9e68-58916cb67f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:11.193537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f043a-41fd-498e-9e68-58916cb67f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:11.195084 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f043a-41fd-498e-9e68-58916cb67f8d service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:11.204221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 730/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:24:13.530491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61ac1a-8db4-4ffc-99a2-6934aecfb534 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:13.531152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61ac1a-8db4-4ffc-99a2-6934aecfb534 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:13.531152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61ac1a-8db4-4ffc-99a2-6934aecfb534 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:13.532438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 758/1488] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:24:13.537660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f5a87-30e4-4aa4-b71e-504f1ec78124 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:13.538100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f5a87-30e4-4aa4-b71e-504f1ec78124 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:13.538205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f5a87-30e4-4aa4-b71e-504f1ec78124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:13.556204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ab7f5a87-30e4-4aa4-b71e-504f1ec78124 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:24:13.557433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ab7f5a87-30e4-4aa4-b71e-504f1ec78124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:13.565184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 731/1489] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:24:13.606259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61ac1a-8db4-4ffc-99a2-6934aecfb534 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:13.607947 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fc61ac1a-8db4-4ffc-99a2-6934aecfb534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:13.621475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61ac1a-8db4-4ffc-99a2-6934aecfb534 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:13.625054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11419a9c-ccc4-4e17-9b45-1744c8eb6c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:13.625783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11419a9c-ccc4-4e17-9b45-1744c8eb6c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:13.626107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11419a9c-ccc4-4e17-9b45-1744c8eb6c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:13.626435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11419a9c-ccc4-4e17-9b45-1744c8eb6c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:13.636814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11419a9c-ccc4-4e17-9b45-1744c8eb6c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:13.638265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11419a9c-ccc4-4e17-9b45-1744c8eb6c39 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:13.647569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 759/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:24:16.076616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7f5a2-249b-432b-b18b-b37e5339777a None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:16.077224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7f5a2-249b-432b-b18b-b37e5339777a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:16.077224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7f5a2-249b-432b-b18b-b37e5339777a None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:16.078597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 732/1491] 10.222.0.30 () {54 vars in 926 bytes} [Mon May 23 15:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:24:16.084631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a099503-6435-4c41-9fe9-04df6fcf0a00 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:16.085095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a099503-6435-4c41-9fe9-04df6fcf0a00 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:16.085261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a099503-6435-4c41-9fe9-04df6fcf0a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:16.102680 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-8a099503-6435-4c41-9fe9-04df6fcf0a00 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:24:16.103864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8a099503-6435-4c41-9fe9-04df6fcf0a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:16.111549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 760/1492] 10.222.0.30 () {58 vars in 1023 bytes} [Mon May 23 15:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:24:16.151671 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7f5a2-249b-432b-b18b-b37e5339777a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:16.162436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7f5a2-249b-432b-b18b-b37e5339777a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:16.165959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305a7e8-f341-48e3-b4b4-098e713112a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:16.166592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305a7e8-f341-48e3-b4b4-098e713112a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:16.166873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305a7e8-f341-48e3-b4b4-098e713112a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:16.167147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305a7e8-f341-48e3-b4b4-098e713112a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:16.175865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305a7e8-f341-48e3-b4b4-098e713112a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:16.177392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305a7e8-f341-48e3-b4b4-098e713112a7 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:16.186648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 733/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:24:26.097489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=33 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:24:26.182528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=33 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:56.416665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24397-ecd4-4750-9bca-e2b879b25c88 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.417305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24397-ecd4-4750-9bca-e2b879b25c88 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.417305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24397-ecd4-4750-9bca-e2b879b25c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.436094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-aac24397-ecd4-4750-9bca-e2b879b25c88 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:56.437317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-aac24397-ecd4-4750-9bca-e2b879b25c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:56.445092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 761/1494] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:25:56.454999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005451d-c7c5-4465-a7a0-1758075218a6 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.455532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005451d-c7c5-4465-a7a0-1758075218a6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.455532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005451d-c7c5-4465-a7a0-1758075218a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.472920 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8005451d-c7c5-4465-a7a0-1758075218a6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:56.474105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8005451d-c7c5-4465-a7a0-1758075218a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:56.482020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 734/1495] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:25:56.486456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac24397-ecd4-4750-9bca-e2b879b25c88 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:56.496767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac24397-ecd4-4750-9bca-e2b879b25c88 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:56.501277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:56.501929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.502211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.502487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.503201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:56.503504 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:56.504502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING py.warnings [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:25:56.504502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: warnings.warn(msg) May 23 15:25:56.504502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: May 23 15:25:56.505418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:56.517569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 762/1496] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:56.525816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005451d-c7c5-4465-a7a0-1758075218a6 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:56.535053 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005451d-c7c5-4465-a7a0-1758075218a6 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:56.539495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee93c-63b9-48de-af9f-3ca6756f9db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:56.540137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee93c-63b9-48de-af9f-3ca6756f9db6 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.540419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee93c-63b9-48de-af9f-3ca6756f9db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.540690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee93c-63b9-48de-af9f-3ca6756f9db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.541956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ee93c-63b9-48de-af9f-3ca6756f9db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:56.543837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WARNING py.warnings [None req-ec3ee93c-63b9-48de-af9f-3ca6756f9db6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:25:56.543837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: warnings.warn(msg) May 23 15:25:56.543837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: May 23 15:25:56.545063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ee93c-63b9-48de-af9f-3ca6756f9db6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:56.581099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 735/1497] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:25:56 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:25:56.585454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:56.595607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b14f40-8211-48dc-9834-efdfc367c620 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:56.653665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425d447-7e12-41e3-b241-2fb3f980a4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:56.654368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425d447-7e12-41e3-b241-2fb3f980a4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.654651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425d447-7e12-41e3-b241-2fb3f980a4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.654927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425d447-7e12-41e3-b241-2fb3f980a4a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.655653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425d447-7e12-41e3-b241-2fb3f980a4a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:56.656699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING py.warnings [None req-c425d447-7e12-41e3-b241-2fb3f980a4a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:25:56.656699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: warnings.warn(msg) May 23 15:25:56.656699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: May 23 15:25:56.657949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425d447-7e12-41e3-b241-2fb3f980a4a0 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:56.677632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 763/1498] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:25:56 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:25:56.681749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee93c-63b9-48de-af9f-3ca6756f9db6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:56.692593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee93c-63b9-48de-af9f-3ca6756f9db6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:56.697188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101f1ae-e9f8-4d37-8a22-a12a819dae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:56.697915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101f1ae-e9f8-4d37-8a22-a12a819dae51 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.698162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101f1ae-e9f8-4d37-8a22-a12a819dae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.698436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101f1ae-e9f8-4d37-8a22-a12a819dae51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.699152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101f1ae-e9f8-4d37-8a22-a12a819dae51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:56.700151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WARNING py.warnings [None req-c101f1ae-e9f8-4d37-8a22-a12a819dae51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:25:56.700151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: warnings.warn(msg) May 23 15:25:56.700151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: May 23 15:25:56.701064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101f1ae-e9f8-4d37-8a22-a12a819dae51 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:56.713665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 736/1499] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:25:56.717559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425d447-7e12-41e3-b241-2fb3f980a4a0 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:56.727176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425d447-7e12-41e3-b241-2fb3f980a4a0 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:56.732673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36faa3-02f0-49d3-8727-d16f12a5890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:56.733399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36faa3-02f0-49d3-8727-d16f12a5890f admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.733692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36faa3-02f0-49d3-8727-d16f12a5890f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.733966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36faa3-02f0-49d3-8727-d16f12a5890f admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.734690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36faa3-02f0-49d3-8727-d16f12a5890f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:56.735694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING py.warnings [None req-9b36faa3-02f0-49d3-8727-d16f12a5890f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:25:56.735694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: warnings.warn(msg) May 23 15:25:56.735694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: May 23 15:25:56.736585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36faa3-02f0-49d3-8727-d16f12a5890f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:56.750843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 764/1500] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:25:56.754871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101f1ae-e9f8-4d37-8a22-a12a819dae51 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:56.764570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101f1ae-e9f8-4d37-8a22-a12a819dae51 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:56.769333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901d147-f123-4942-8732-0605ed64c506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:56.770198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901d147-f123-4942-8732-0605ed64c506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.770478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901d147-f123-4942-8732-0605ed64c506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.770753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901d147-f123-4942-8732-0605ed64c506 admin admin] PATH_INFO: `/v3/projects/aca03283bd22471197645a5860f70dc7/users/e5d5ec3970bd4898b122f9370e907b15/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.785284 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901d147-f123-4942-8732-0605ed64c506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca03283bd22471197645a5860f70dc7, user_id=e5d5ec3970bd4898b122f9370e907b15, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:56.786319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WARNING py.warnings [None req-9901d147-f123-4942-8732-0605ed64c506 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:25:56.786319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: warnings.warn(msg) May 23 15:25:56.786319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: May 23 15:25:56.788249 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901d147-f123-4942-8732-0605ed64c506 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:56.801985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 737/1501] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:25:56 2022] PUT /identity/v3/projects/aca03283bd22471197645a5860f70dc7/users/e5d5ec3970bd4898b122f9370e907b15/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:25:56.813857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fa1c5-e6c0-4b96-9139-f4529655070e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.814145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fa1c5-e6c0-4b96-9139-f4529655070e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.814421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fa1c5-e6c0-4b96-9139-f4529655070e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.839268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-339fa1c5-e6c0-4b96-9139-f4529655070e None None] MFA Rules not processed for user `e5d5ec3970bd4898b122f9370e907b15`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:56.840452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-339fa1c5-e6c0-4b96-9139-f4529655070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:56.903038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 765/1502] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:25:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:25:56.916209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378dada-bf64-4d83-8d2b-7108be9b7fbf None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.917051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378dada-bf64-4d83-8d2b-7108be9b7fbf None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.917325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378dada-bf64-4d83-8d2b-7108be9b7fbf None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.918869 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 738/1503] 10.222.0.30 () {54 vars in 963 bytes} [Mon May 23 15:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:56.923609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19346a58-a098-4b20-b759-3e0b1a770c19 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:56.923898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19346a58-a098-4b20-b759-3e0b1a770c19 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:56.924177 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19346a58-a098-4b20-b759-3e0b1a770c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:56.942278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-19346a58-a098-4b20-b759-3e0b1a770c19 None None] MFA Rules not processed for user `36daf80a4cbc4cfc8962c7924433d257`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:56.943427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-19346a58-a098-4b20-b759-3e0b1a770c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:56.987789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:56.996153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 766/1504] 10.222.0.30 () {58 vars in 1060 bytes} [Mon May 23 15:25:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:25:57.003805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378dada-bf64-4d83-8d2b-7108be9b7fbf None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:57.014313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378dada-bf64-4d83-8d2b-7108be9b7fbf None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:57.017856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db2400-87fd-4895-8878-3e117b6c7c4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:57.018744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2400-87fd-4895-8878-3e117b6c7c4c service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:57.019027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2400-87fd-4895-8878-3e117b6c7c4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:57.019303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2400-87fd-4895-8878-3e117b6c7c4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:57.028200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db2400-87fd-4895-8878-3e117b6c7c4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:57.029686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db2400-87fd-4895-8878-3e117b6c7c4c service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:57.038305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 739/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:25:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:25:57.080067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a57de-8a8a-4154-988d-886520a05252 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:57.080355 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a57de-8a8a-4154-988d-886520a05252 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:57.080635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a57de-8a8a-4154-988d-886520a05252 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:57.081992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 767/1506] 10.222.0.30 () {52 vars in 829 bytes} [Mon May 23 15:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:57.085599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db2400-87fd-4895-8878-3e117b6c7c4c service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:57.149743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db2400-87fd-4895-8878-3e117b6c7c4c service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:57.154704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:57.155568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:57.155844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:57.156115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] PATH_INFO: `/v3/users/e5d5ec3970bd4898b122f9370e907b15` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:57.157299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d5ec3970bd4898b122f9370e907b15)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:57.158336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WARNING py.warnings [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:25:57.158336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: warnings.warn(msg) May 23 15:25:57.158336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: May 23 15:25:57.159542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:57.224573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d5ec3970bd4898b122f9370e907b15'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.225256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d5ec3970bd4898b122f9370e907b15'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.238819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d5ec3970bd4898b122f9370e907b15'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.239497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d5ec3970bd4898b122f9370e907b15'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.257438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d5ec3970bd4898b122f9370e907b15'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.261489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:57.274368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 740/1507] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:25:57 2022] DELETE /identity/v3/users/e5d5ec3970bd4898b122f9370e907b15 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:25:57.279368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f95d8-d003-4269-9b57-37f31c3181c5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:57.279657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f95d8-d003-4269-9b57-37f31c3181c5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:57.279936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f95d8-d003-4269-9b57-37f31c3181c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:57.297746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-404f95d8-d003-4269-9b57-37f31c3181c5 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:57.298965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-404f95d8-d003-4269-9b57-37f31c3181c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:57.356326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 768/1508] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:25:57.366653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:57.368075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:57.427489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d1311-45f0-46f0-83d5-56f43e88e316 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:57.431076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd13f3-6fe6-4bfd-a795-00bc582cabcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:57.431718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd13f3-6fe6-4bfd-a795-00bc582cabcc service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:57.432000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd13f3-6fe6-4bfd-a795-00bc582cabcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:57.432364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd13f3-6fe6-4bfd-a795-00bc582cabcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:57.441189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd13f3-6fe6-4bfd-a795-00bc582cabcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:57.442638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd13f3-6fe6-4bfd-a795-00bc582cabcc service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:57.452555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 741/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:25:57.867604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f95d8-d003-4269-9b57-37f31c3181c5 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:57.869039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-404f95d8-d003-4269-9b57-37f31c3181c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:57.880859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f95d8-d003-4269-9b57-37f31c3181c5 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:57.885616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:57.887239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:57.887528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:57.887798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] PATH_INFO: `/v3/projects/aca03283bd22471197645a5860f70dc7` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:57.889216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca03283bd22471197645a5860f70dc7)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:57.890107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING py.warnings [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:25:57.890107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: warnings.warn(msg) May 23 15:25:57.890107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: May 23 15:25:57.891576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:57.957730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca03283bd22471197645a5860f70dc7', 'request_id': 'req-46b6995c-e084-4de4-8808-b0c957a7fc9f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.971813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca03283bd22471197645a5860f70dc7', 'request_id': 'req-46b6995c-e084-4de4-8808-b0c957a7fc9f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.977906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca03283bd22471197645a5860f70dc7', 'request_id': 'req-46b6995c-e084-4de4-8808-b0c957a7fc9f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.992541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] The token cache is being invalidate because project aca03283bd22471197645a5860f70dc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:25:57.992828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.993480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-46b6995c-e084-4de4-8808-b0c957a7fc9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:25:57.995292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 769/1510] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:25:57 2022] DELETE /identity/v3/projects/aca03283bd22471197645a5860f70dc7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:26:07.447831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:26:07.989617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:27:40.463182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5fbd8-492b-4167-a076-661cb957e20a None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:40.463779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5fbd8-492b-4167-a076-661cb957e20a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:40.463779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5fbd8-492b-4167-a076-661cb957e20a None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:40.465164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 742/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:27:40.469973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c70de4-2ca5-46fc-9e5e-d612b63f5704 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:40.470404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c70de4-2ca5-46fc-9e5e-d612b63f5704 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:40.470568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c70de4-2ca5-46fc-9e5e-d612b63f5704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:40.488544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-29c70de4-2ca5-46fc-9e5e-d612b63f5704 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:27:40.489809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-29c70de4-2ca5-46fc-9e5e-d612b63f5704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:27:40.548526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 770/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:27:40.721020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5fbd8-492b-4167-a076-661cb957e20a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:27:40.722819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e1a5fbd8-492b-4167-a076-661cb957e20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:27:40.795354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5fbd8-492b-4167-a076-661cb957e20a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:27:40.800585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfa60f-1679-4cb9-a9f0-fcb74000e922 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:27:40.801253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfa60f-1679-4cb9-a9f0-fcb74000e922 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:40.801531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfa60f-1679-4cb9-a9f0-fcb74000e922 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:40.801806 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfa60f-1679-4cb9-a9f0-fcb74000e922 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:40.810708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfa60f-1679-4cb9-a9f0-fcb74000e922 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:27:40.812311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfa60f-1679-4cb9-a9f0-fcb74000e922 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:27:40.822176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 743/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:27:41.581891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08e6e0-d1a9-45ab-b1d3-b4eabd27d5a0 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:41.582458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08e6e0-d1a9-45ab-b1d3-b4eabd27d5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:41.582458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08e6e0-d1a9-45ab-b1d3-b4eabd27d5a0 None None] PATH_INFO: `/v3` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:41.583789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 771/1514] 10.222.0.72 () {54 vars in 925 bytes} [Mon May 23 15:27:41 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:27:41.592088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87879ce4-7882-4e06-aee0-a7cb8a2bbf27 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:41.592551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87879ce4-7882-4e06-aee0-a7cb8a2bbf27 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:41.592656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87879ce4-7882-4e06-aee0-a7cb8a2bbf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:41.611198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-87879ce4-7882-4e06-aee0-a7cb8a2bbf27 None None] MFA Rules not processed for user `3a7dd5a0102d4d3493f09e809cb3caaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:27:41.612392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-87879ce4-7882-4e06-aee0-a7cb8a2bbf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:27:41.676851 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 744/1515] 10.222.0.72 () {58 vars in 1015 bytes} [Mon May 23 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:27:41.721943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486af72-a962-4249-817c-9e21b006450f None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:41.722229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486af72-a962-4249-817c-9e21b006450f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:41.722507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486af72-a962-4249-817c-9e21b006450f None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:41.723879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 772/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:27:41.728546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828bff1-536b-4939-81a2-74977ea45930 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:41.728835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828bff1-536b-4939-81a2-74977ea45930 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:41.729139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828bff1-536b-4939-81a2-74977ea45930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:41.746802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e828bff1-536b-4939-81a2-74977ea45930 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:27:41.747976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e828bff1-536b-4939-81a2-74977ea45930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:27:41.755706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 745/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:27:41.906934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486af72-a962-4249-817c-9e21b006450f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:27:41.921733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486af72-a962-4249-817c-9e21b006450f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:27:41.927043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc607159-0a21-471f-97b0-3820c1386e6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:27:41.927681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc607159-0a21-471f-97b0-3820c1386e6e service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:41.928026 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc607159-0a21-471f-97b0-3820c1386e6e service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:41.928384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc607159-0a21-471f-97b0-3820c1386e6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:41.936634 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc607159-0a21-471f-97b0-3820c1386e6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:27:41.938310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc607159-0a21-471f-97b0-3820c1386e6e service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:27:41.947801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 773/1518] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:27:50.609699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6112-a8f9-440c-a628-5e9bf9d71e6f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:50.610095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6112-a8f9-440c-a628-5e9bf9d71e6f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:50.610251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6112-a8f9-440c-a628-5e9bf9d71e6f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:50.611641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 746/1519] 10.222.0.30 () {54 vars in 960 bytes} [Mon May 23 15:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:27:50.616713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af88c7-906a-47da-a468-662e03ea9345 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:50.617154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af88c7-906a-47da-a468-662e03ea9345 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:50.617313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af88c7-906a-47da-a468-662e03ea9345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:50.635749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d7af88c7-906a-47da-a468-662e03ea9345 None None] MFA Rules not processed for user `3a7dd5a0102d4d3493f09e809cb3caaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:27:50.637008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d7af88c7-906a-47da-a468-662e03ea9345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:27:50.646525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 774/1520] 10.222.0.30 () {58 vars in 1057 bytes} [Mon May 23 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:27:50.653675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a6112-a8f9-440c-a628-5e9bf9d71e6f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:27:50.667372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a6112-a8f9-440c-a628-5e9bf9d71e6f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:27:50.672229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d7ce9-77db-45cc-9a6e-d1f97eca480b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:27:50.672881 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d7ce9-77db-45cc-9a6e-d1f97eca480b service placement] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:50.673172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d7ce9-77db-45cc-9a6e-d1f97eca480b service placement] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:50.673452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d7ce9-77db-45cc-9a6e-d1f97eca480b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:50.682405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d7ce9-77db-45cc-9a6e-d1f97eca480b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:27:50.684044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d7ce9-77db-45cc-9a6e-d1f97eca480b service placement] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:27:50.695900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 747/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:27:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:27:53.201424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d31ff8-6d7c-41e9-8280-0c0b9218197f None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:53.202043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d31ff8-6d7c-41e9-8280-0c0b9218197f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:53.202043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d31ff8-6d7c-41e9-8280-0c0b9218197f None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:53.203272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 775/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:27:53.208656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cbb0f-5302-41e9-85a0-44800f3d0f86 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:53.209111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cbb0f-5302-41e9-85a0-44800f3d0f86 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:53.209349 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cbb0f-5302-41e9-85a0-44800f3d0f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:53.227550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-927cbb0f-5302-41e9-85a0-44800f3d0f86 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:27:53.228731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-927cbb0f-5302-41e9-85a0-44800f3d0f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:27:53.236425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 748/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:27:53.388267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d31ff8-6d7c-41e9-8280-0c0b9218197f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:27:53.403521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d31ff8-6d7c-41e9-8280-0c0b9218197f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:27:53.408847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d60b43-a7f3-4d41-8088-c1c2bf17208a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:27:53.409507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d60b43-a7f3-4d41-8088-c1c2bf17208a service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:53.409789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d60b43-a7f3-4d41-8088-c1c2bf17208a service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:53.410069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d60b43-a7f3-4d41-8088-c1c2bf17208a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:53.418935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d60b43-a7f3-4d41-8088-c1c2bf17208a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:27:53.420514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d60b43-a7f3-4d41-8088-c1c2bf17208a service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:27:53.430140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 776/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:27:54.356201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e1443-f84a-4c2a-ab60-5520e8d30fe6 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:54.356856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e1443-f84a-4c2a-ab60-5520e8d30fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:54.356856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e1443-f84a-4c2a-ab60-5520e8d30fe6 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:54.358316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 749/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:27:54.363124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ba2b9-6419-4fa4-b955-c532eb1bb049 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:54.363575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ba2b9-6419-4fa4-b955-c532eb1bb049 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:54.363716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ba2b9-6419-4fa4-b955-c532eb1bb049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:54.383013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d29ba2b9-6419-4fa4-b955-c532eb1bb049 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:27:54.384211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d29ba2b9-6419-4fa4-b955-c532eb1bb049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:27:54.392541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 777/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:27:54.544531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e1443-f84a-4c2a-ab60-5520e8d30fe6 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:27:54.559383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e1443-f84a-4c2a-ab60-5520e8d30fe6 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:27:54.564947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787e1b6-4a4c-41b2-ad71-3190ce5c8e27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:27:54.565586 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787e1b6-4a4c-41b2-ad71-3190ce5c8e27 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:54.565870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787e1b6-4a4c-41b2-ad71-3190ce5c8e27 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:54.566164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787e1b6-4a4c-41b2-ad71-3190ce5c8e27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:54.574870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787e1b6-4a4c-41b2-ad71-3190ce5c8e27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:27:54.576482 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787e1b6-4a4c-41b2-ad71-3190ce5c8e27 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:27:54.586395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 750/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:27:58.971551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4533b6-e03c-414c-abaf-92ad7248ff0f None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:58.972106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4533b6-e03c-414c-abaf-92ad7248ff0f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:58.972106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4533b6-e03c-414c-abaf-92ad7248ff0f None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:58.973960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 778/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:27:58.979113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc405-c68f-454f-8667-99742e222465 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:58.979554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc405-c68f-454f-8667-99742e222465 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:58.979663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc405-c68f-454f-8667-99742e222465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:58.997919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-69fdc405-c68f-454f-8667-99742e222465 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:27:58.999096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-69fdc405-c68f-454f-8667-99742e222465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:27:59.007299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 751/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:27:59.045301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4533b6-e03c-414c-abaf-92ad7248ff0f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:27:59.046974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-bb4533b6-e03c-414c-abaf-92ad7248ff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:27:59.105036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4533b6-e03c-414c-abaf-92ad7248ff0f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:27:59.108527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98567480-3e2b-4b2c-a145-30855b3a9366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:27:59.109208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98567480-3e2b-4b2c-a145-30855b3a9366 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:27:59.109487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98567480-3e2b-4b2c-a145-30855b3a9366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:27:59.109761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98567480-3e2b-4b2c-a145-30855b3a9366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:27:59.119211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98567480-3e2b-4b2c-a145-30855b3a9366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:27:59.120686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98567480-3e2b-4b2c-a145-30855b3a9366 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:27:59.131022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 779/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:28:01.327890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3a799-df7e-42e7-ba13-c7dffa77142c None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:28:01.328483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3a799-df7e-42e7-ba13-c7dffa77142c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:28:01.328483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3a799-df7e-42e7-ba13-c7dffa77142c None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:28:01.330140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 752/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:28:01.336114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0f20b-e9c5-4ed4-b9e1-3b2b4ca5d2cd None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:28:01.336550 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0f20b-e9c5-4ed4-b9e1-3b2b4ca5d2cd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:28:01.336649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0f20b-e9c5-4ed4-b9e1-3b2b4ca5d2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:28:01.354415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-78a0f20b-e9c5-4ed4-b9e1-3b2b4ca5d2cd None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:28:01.355590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-78a0f20b-e9c5-4ed4-b9e1-3b2b4ca5d2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:28:01.363310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 780/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:28:01.400676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3a799-df7e-42e7-ba13-c7dffa77142c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:28:01.415611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3a799-df7e-42e7-ba13-c7dffa77142c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:28:01.419517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37d30f-026f-404c-80c0-f874cf566712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:28:01.420454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37d30f-026f-404c-80c0-f874cf566712 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:28:01.420753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37d30f-026f-404c-80c0-f874cf566712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:28:01.421130 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37d30f-026f-404c-80c0-f874cf566712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:28:01.433702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37d30f-026f-404c-80c0-f874cf566712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:28:01.435524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37d30f-026f-404c-80c0-f874cf566712 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:28:01.448208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 753/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:28:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:28:05.206303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1d199-8199-4db0-8197-cdd5c78d5f36 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:28:05.206862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1d199-8199-4db0-8197-cdd5c78d5f36 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:28:05.206862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1d199-8199-4db0-8197-cdd5c78d5f36 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:28:05.208827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 781/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:28:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:28:05.214322 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fca6b-3f2e-49e8-ae2c-35bacb6cd88f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:28:05.214757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fca6b-3f2e-49e8-ae2c-35bacb6cd88f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:28:05.214868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fca6b-3f2e-49e8-ae2c-35bacb6cd88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:28:05.235459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-223fca6b-3f2e-49e8-ae2c-35bacb6cd88f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:28:05.236671 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-223fca6b-3f2e-49e8-ae2c-35bacb6cd88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:28:05.244432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 754/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:28:05.523373 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1d199-8199-4db0-8197-cdd5c78d5f36 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:28:05.525077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8db1d199-8199-4db0-8197-cdd5c78d5f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:28:05.585083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1d199-8199-4db0-8197-cdd5c78d5f36 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:28:05.588705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf1838-9b88-4b79-a8d6-3486875531c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:28:05.589405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf1838-9b88-4b79-a8d6-3486875531c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:28:05.589699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf1838-9b88-4b79-a8d6-3486875531c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:28:05.589983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf1838-9b88-4b79-a8d6-3486875531c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:28:05.600399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf1838-9b88-4b79-a8d6-3486875531c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:28:05.601887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf1838-9b88-4b79-a8d6-3486875531c5 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:28:05.607317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:28:05.615300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 782/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:28:07.871161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eba7b-ea90-4218-81ac-fa7804e57389 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:28:07.871699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eba7b-ea90-4218-81ac-fa7804e57389 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:28:07.871699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eba7b-ea90-4218-81ac-fa7804e57389 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:28:07.873055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 755/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:28:07.878525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160262e-e22e-40b6-b28f-5af75b63d09a None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:28:07.879555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160262e-e22e-40b6-b28f-5af75b63d09a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:28:07.879746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160262e-e22e-40b6-b28f-5af75b63d09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:28:07.898128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0160262e-e22e-40b6-b28f-5af75b63d09a None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:28:07.899342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0160262e-e22e-40b6-b28f-5af75b63d09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:28:07.907317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 783/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:28:08.179932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eba7b-ea90-4218-81ac-fa7804e57389 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:28:08.181634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-cc1eba7b-ea90-4218-81ac-fa7804e57389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:28:08.189209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=94 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:28:08.196090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eba7b-ea90-4218-81ac-fa7804e57389 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:28:08.199744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5601dca-7da8-465b-a627-02a28fd3082a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:28:08.200377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5601dca-7da8-465b-a627-02a28fd3082a service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:28:08.200657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5601dca-7da8-465b-a627-02a28fd3082a service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:28:08.200982 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5601dca-7da8-465b-a627-02a28fd3082a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:28:08.209870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5601dca-7da8-465b-a627-02a28fd3082a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:28:08.211310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5601dca-7da8-465b-a627-02a28fd3082a service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:28:08.220810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 756/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:28:17.892649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:28:18.216717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:42.273018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b57db-26ea-4561-99a0-632d4bf156b1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.274289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b57db-26ea-4561-99a0-632d4bf156b1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.274289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b57db-26ea-4561-99a0-632d4bf156b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.289787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940329-09d6-4b8f-8d6c-1dbd37e6f124 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.290378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940329-09d6-4b8f-8d6c-1dbd37e6f124 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.290378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940329-09d6-4b8f-8d6c-1dbd37e6f124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.309206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-583b57db-26ea-4561-99a0-632d4bf156b1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.310501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-583b57db-26ea-4561-99a0-632d4bf156b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.322311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-86940329-09d6-4b8f-8d6c-1dbd37e6f124 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.323563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-86940329-09d6-4b8f-8d6c-1dbd37e6f124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.327966 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 784/1540] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.329477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f0e2-cb3d-4fe2-b3e2-907f88c11631 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.329771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f0e2-cb3d-4fe2-b3e2-907f88c11631 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.330179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f0e2-cb3d-4fe2-b3e2-907f88c11631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.350042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 757/1541] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.351575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc051c-69a9-40c3-87ee-69b9362bc195 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.351870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc051c-69a9-40c3-87ee-69b9362bc195 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.352145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc051c-69a9-40c3-87ee-69b9362bc195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.358090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f4d1f0e2-cb3d-4fe2-b3e2-907f88c11631 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.359438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f4d1f0e2-cb3d-4fe2-b3e2-907f88c11631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.376117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 785/1542] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.377580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92916645-9c1d-44dd-9bc6-518c5c0dc3ed None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.377873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92916645-9c1d-44dd-9bc6-518c5c0dc3ed None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.378152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92916645-9c1d-44dd-9bc6-518c5c0dc3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.389541 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-c7cc051c-69a9-40c3-87ee-69b9362bc195 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.390778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c7cc051c-69a9-40c3-87ee-69b9362bc195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.413004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 758/1543] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.414331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cfb4c-6444-4915-80ed-481b6f6fcd16 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.414617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cfb4c-6444-4915-80ed-481b6f6fcd16 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.414893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cfb4c-6444-4915-80ed-481b6f6fcd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.417814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-92916645-9c1d-44dd-9bc6-518c5c0dc3ed None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.419019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-92916645-9c1d-44dd-9bc6-518c5c0dc3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.441097 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 786/1544] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.442639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb74b78-3f79-450d-b2d7-b5c6e06d001e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.442932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb74b78-3f79-450d-b2d7-b5c6e06d001e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.443212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb74b78-3f79-450d-b2d7-b5c6e06d001e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.452296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-914cfb4c-6444-4915-80ed-481b6f6fcd16 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.453926 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-914cfb4c-6444-4915-80ed-481b6f6fcd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.463326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 759/1545] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.464266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-7bb74b78-3f79-450d-b2d7-b5c6e06d001e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.465234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4a0dd-231c-44d8-b91d-fc1085a83107 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.465640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4a0dd-231c-44d8-b91d-fc1085a83107 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.466028 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4a0dd-231c-44d8-b91d-fc1085a83107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.466188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7bb74b78-3f79-450d-b2d7-b5c6e06d001e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.478444 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 787/1546] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.480100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb74b78-3f79-450d-b2d7-b5c6e06d001e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.491358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb74b78-3f79-450d-b2d7-b5c6e06d001e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:42.492627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a4b4a0dd-231c-44d8-b91d-fc1085a83107 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.494180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a4b4a0dd-231c-44d8-b91d-fc1085a83107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.497471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bbe06-12a6-4feb-8b73-64bf9460ce61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:42.498822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bbe06-12a6-4feb-8b73-64bf9460ce61 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.499110 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bbe06-12a6-4feb-8b73-64bf9460ce61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.499390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bbe06-12a6-4feb-8b73-64bf9460ce61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.500486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bbe06-12a6-4feb-8b73-64bf9460ce61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:42.500797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bbe06-12a6-4feb-8b73-64bf9460ce61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:42.502423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bbe06-12a6-4feb-8b73-64bf9460ce61 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:42.504001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 760/1547] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.505435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f258b6-ffc1-4faf-a2ed-1b18bde7a8e7 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.505724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f258b6-ffc1-4faf-a2ed-1b18bde7a8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.506002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f258b6-ffc1-4faf-a2ed-1b18bde7a8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.516490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 788/1548] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:29:42.517818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518310b-da6b-444b-b38c-c080d78a343f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.518113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518310b-da6b-444b-b38c-c080d78a343f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.518390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518310b-da6b-444b-b38c-c080d78a343f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.526175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d9f258b6-ffc1-4faf-a2ed-1b18bde7a8e7 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.527540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d9f258b6-ffc1-4faf-a2ed-1b18bde7a8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.536227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-9518310b-da6b-444b-b38c-c080d78a343f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.536896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 761/1549] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.545036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9518310b-da6b-444b-b38c-c080d78a343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.545149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14597ee-f4b9-4e54-a28b-b3c035ea8119 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.545149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14597ee-f4b9-4e54-a28b-b3c035ea8119 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.545149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14597ee-f4b9-4e54-a28b-b3c035ea8119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.551337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 789/1550] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.552302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518310b-da6b-444b-b38c-c080d78a343f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.562057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518310b-da6b-444b-b38c-c080d78a343f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:42.562924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d14597ee-f4b9-4e54-a28b-b3c035ea8119 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.564225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d14597ee-f4b9-4e54-a28b-b3c035ea8119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.566552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489bf97-dcc1-40e6-a001-497709fff7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:42.567192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489bf97-dcc1-40e6-a001-497709fff7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.567473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489bf97-dcc1-40e6-a001-497709fff7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.567746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489bf97-dcc1-40e6-a001-497709fff7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.568454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489bf97-dcc1-40e6-a001-497709fff7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:42.568754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489bf97-dcc1-40e6-a001-497709fff7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:42.570363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489bf97-dcc1-40e6-a001-497709fff7ea admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:42.572347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 762/1551] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.573752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4689b-4a21-40da-9c8b-8d39b66f4e27 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.574047 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4689b-4a21-40da-9c8b-8d39b66f4e27 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.574325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4689b-4a21-40da-9c8b-8d39b66f4e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.589895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 790/1552] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:29:42.589895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22314c99-e8c4-4019-9b92-4fa641bd9b69 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.589895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22314c99-e8c4-4019-9b92-4fa641bd9b69 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.589895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22314c99-e8c4-4019-9b92-4fa641bd9b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.595409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d5a4689b-4a21-40da-9c8b-8d39b66f4e27 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.596673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d5a4689b-4a21-40da-9c8b-8d39b66f4e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.605090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 763/1553] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.605967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4689b-4a21-40da-9c8b-8d39b66f4e27 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.610811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-22314c99-e8c4-4019-9b92-4fa641bd9b69 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.612203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-22314c99-e8c4-4019-9b92-4fa641bd9b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.619788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4689b-4a21-40da-9c8b-8d39b66f4e27 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:42.625092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00220493-1539-492b-bc67-57db96dc4044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:42.625694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 791/1554] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.625810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00220493-1539-492b-bc67-57db96dc4044 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.626084 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00220493-1539-492b-bc67-57db96dc4044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.626334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00220493-1539-492b-bc67-57db96dc4044 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.626685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22314c99-e8c4-4019-9b92-4fa641bd9b69 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.627116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00220493-1539-492b-bc67-57db96dc4044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:42.627364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00220493-1539-492b-bc67-57db96dc4044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:42.628396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WARNING py.warnings [None req-00220493-1539-492b-bc67-57db96dc4044 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:29:42.628396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: warnings.warn(msg) May 23 15:29:42.628396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: May 23 15:29:42.629335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00220493-1539-492b-bc67-57db96dc4044 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:42.637893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22314c99-e8c4-4019-9b92-4fa641bd9b69 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:42.644685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e00a34-b8b3-4ec4-8aa2-b1ef9d903f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:42.645522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e00a34-b8b3-4ec4-8aa2-b1ef9d903f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.645808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e00a34-b8b3-4ec4-8aa2-b1ef9d903f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.646087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e00a34-b8b3-4ec4-8aa2-b1ef9d903f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.646791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e00a34-b8b3-4ec4-8aa2-b1ef9d903f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:42.647171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 764/1555] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:29:42.648004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING py.warnings [None req-66e00a34-b8b3-4ec4-8aa2-b1ef9d903f91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:29:42.648004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: warnings.warn(msg) May 23 15:29:42.648004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: May 23 15:29:42.648589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98ca6-7c59-41c1-bc83-bf4144384a98 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.648972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98ca6-7c59-41c1-bc83-bf4144384a98 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.649256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98ca6-7c59-41c1-bc83-bf4144384a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.649419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e00a34-b8b3-4ec4-8aa2-b1ef9d903f91 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:42.669304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-c4d98ca6-7c59-41c1-bc83-bf4144384a98 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.671384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c4d98ca6-7c59-41c1-bc83-bf4144384a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.680364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 765/1556] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.681213 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d98ca6-7c59-41c1-bc83-bf4144384a98 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.681813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 792/1557] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:42.682462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e00a34-b8b3-4ec4-8aa2-b1ef9d903f91 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.693904 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d98ca6-7c59-41c1-bc83-bf4144384a98 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:42.746932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e00a34-b8b3-4ec4-8aa2-b1ef9d903f91 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:42.753601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a8305-4bf7-4fa0-82d1-bce6f7fa2bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:42.754134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1dcc2-b61c-4e07-b532-ff1f7dd59898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:42.754278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a8305-4bf7-4fa0-82d1-bce6f7fa2bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.754561 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a8305-4bf7-4fa0-82d1-bce6f7fa2bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.754820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1dcc2-b61c-4e07-b532-ff1f7dd59898 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.754922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a8305-4bf7-4fa0-82d1-bce6f7fa2bdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.755233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1dcc2-b61c-4e07-b532-ff1f7dd59898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.755518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1dcc2-b61c-4e07-b532-ff1f7dd59898 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.755612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a8305-4bf7-4fa0-82d1-bce6f7fa2bdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:42.755919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a8305-4bf7-4fa0-82d1-bce6f7fa2bdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:42.756382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1dcc2-b61c-4e07-b532-ff1f7dd59898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:42.756689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1dcc2-b61c-4e07-b532-ff1f7dd59898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:42.757665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a8305-4bf7-4fa0-82d1-bce6f7fa2bdd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:42.758360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1dcc2-b61c-4e07-b532-ff1f7dd59898 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:42.778916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 766/1558] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:29:42.778916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1dcc2-b61c-4e07-b532-ff1f7dd59898 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.779210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 793/1559] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:29:42.779210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04f99d-74c4-4574-b249-63e9123805db None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.779210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04f99d-74c4-4574-b249-63e9123805db None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.779210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04f99d-74c4-4574-b249-63e9123805db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.787143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1dcc2-b61c-4e07-b532-ff1f7dd59898 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:42.791999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d2d70-45f8-40a6-9755-5e928bc56b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:42.792652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d2d70-45f8-40a6-9755-5e928bc56b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.792997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d2d70-45f8-40a6-9755-5e928bc56b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.793276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d2d70-45f8-40a6-9755-5e928bc56b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.793963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d2d70-45f8-40a6-9755-5e928bc56b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:42.795939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d2d70-45f8-40a6-9755-5e928bc56b06 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:42.798873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0c04f99d-74c4-4574-b249-63e9123805db None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.801083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0c04f99d-74c4-4574-b249-63e9123805db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.817764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 794/1560] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.818890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963208e-6828-4982-9a5a-056b159d7e95 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.819285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963208e-6828-4982-9a5a-056b159d7e95 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.819596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963208e-6828-4982-9a5a-056b159d7e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.832324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 767/1561] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:42.833583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d2d70-45f8-40a6-9755-5e928bc56b06 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.842089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6963208e-6828-4982-9a5a-056b159d7e95 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:42.843343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6963208e-6828-4982-9a5a-056b159d7e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:42.895610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:42.897439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d2d70-45f8-40a6-9755-5e928bc56b06 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:42.902542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cd372-7644-4564-b0ee-149bd1f6716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:42.903205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cd372-7644-4564-b0ee-149bd1f6716b admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.903512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cd372-7644-4564-b0ee-149bd1f6716b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.903790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cd372-7644-4564-b0ee-149bd1f6716b admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.904511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cd372-7644-4564-b0ee-149bd1f6716b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:42.905869 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 795/1562] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:42.906490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cd372-7644-4564-b0ee-149bd1f6716b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:42.907126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963208e-6828-4982-9a5a-056b159d7e95 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.917515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:42.917860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963208e-6828-4982-9a5a-056b159d7e95 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:42.923856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a3dd6-b5fa-4604-8ad6-d81a17e3a4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:42.924538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a3dd6-b5fa-4604-8ad6-d81a17e3a4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:42.924832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a3dd6-b5fa-4604-8ad6-d81a17e3a4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:42.925135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a3dd6-b5fa-4604-8ad6-d81a17e3a4d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:42.925866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a3dd6-b5fa-4604-8ad6-d81a17e3a4d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:42.926173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a3dd6-b5fa-4604-8ad6-d81a17e3a4d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:42.927808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a3dd6-b5fa-4604-8ad6-d81a17e3a4d8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:42.935073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 768/1563] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:42.936613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cd372-7644-4564-b0ee-149bd1f6716b admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.946999 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 796/1564] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:29:42.946999 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a3dd6-b5fa-4604-8ad6-d81a17e3a4d8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:42.950660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a3dd6-b5fa-4604-8ad6-d81a17e3a4d8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.011099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3243a2-fe6c-4f86-b1f7-1f290c362d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.011891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3243a2-fe6c-4f86-b1f7-1f290c362d3f admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.012185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3243a2-fe6c-4f86-b1f7-1f290c362d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.012460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3243a2-fe6c-4f86-b1f7-1f290c362d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.013181 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3243a2-fe6c-4f86-b1f7-1f290c362d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.015005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cd372-7644-4564-b0ee-149bd1f6716b admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.015223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3243a2-fe6c-4f86-b1f7-1f290c362d3f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.019787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d714f-b766-440d-93e6-bb8c31bb6baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.020455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d714f-b766-440d-93e6-bb8c31bb6baa admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.020742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d714f-b766-440d-93e6-bb8c31bb6baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.021077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d714f-b766-440d-93e6-bb8c31bb6baa admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.021823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d714f-b766-440d-93e6-bb8c31bb6baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.022876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WARNING py.warnings [None req-c37d714f-b766-440d-93e6-bb8c31bb6baa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:29:43.022876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: warnings.warn(msg) May 23 15:29:43.022876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: May 23 15:29:43.024154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d714f-b766-440d-93e6-bb8c31bb6baa admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.043695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 797/1565] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.044546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3243a2-fe6c-4f86-b1f7-1f290c362d3f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.051560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 769/1566] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:42 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.051560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d714f-b766-440d-93e6-bb8c31bb6baa admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.059657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d714f-b766-440d-93e6-bb8c31bb6baa admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.107048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3243a2-fe6c-4f86-b1f7-1f290c362d3f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.111770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cf4ef-5f6b-4ec6-af7f-96f935148d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.112427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cf4ef-5f6b-4ec6-af7f-96f935148d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.112713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cf4ef-5f6b-4ec6-af7f-96f935148d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.113022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cf4ef-5f6b-4ec6-af7f-96f935148d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.113708 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cf4ef-5f6b-4ec6-af7f-96f935148d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.115246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deb3c8-4dea-4944-acad-5d1b3ae6c364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.115815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cf4ef-5f6b-4ec6-af7f-96f935148d3a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.115937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deb3c8-4dea-4944-acad-5d1b3ae6c364 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.116205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deb3c8-4dea-4944-acad-5d1b3ae6c364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.116446 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deb3c8-4dea-4944-acad-5d1b3ae6c364 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.117204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21deb3c8-4dea-4944-acad-5d1b3ae6c364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:43.117522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21deb3c8-4dea-4944-acad-5d1b3ae6c364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.119117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21deb3c8-4dea-4944-acad-5d1b3ae6c364 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.131526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 770/1567] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:29:43.134027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deb3c8-4dea-4944-acad-5d1b3ae6c364 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.148080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 798/1568] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.148288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deb3c8-4dea-4944-acad-5d1b3ae6c364 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.148396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cf4ef-5f6b-4ec6-af7f-96f935148d3a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.211435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a5d0-4ffa-41fb-9a79-dc9778abaa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.212114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a5d0-4ffa-41fb-9a79-dc9778abaa30 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.212398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a5d0-4ffa-41fb-9a79-dc9778abaa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.212676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a5d0-4ffa-41fb-9a79-dc9778abaa30 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.213427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282a5d0-4ffa-41fb-9a79-dc9778abaa30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.214140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:43.215402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282a5d0-4ffa-41fb-9a79-dc9778abaa30 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.218376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cf4ef-5f6b-4ec6-af7f-96f935148d3a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.223973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5fdb-3cae-4205-b40c-3fbe836d0f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.224838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5fdb-3cae-4205-b40c-3fbe836d0f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.225205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5fdb-3cae-4205-b40c-3fbe836d0f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.225551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5fdb-3cae-4205-b40c-3fbe836d0f0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.226367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf5fdb-3cae-4205-b40c-3fbe836d0f0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:43.226749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf5fdb-3cae-4205-b40c-3fbe836d0f0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.228745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf5fdb-3cae-4205-b40c-3fbe836d0f0e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.238483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 771/1569] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.239767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a5d0-4ffa-41fb-9a79-dc9778abaa30 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.241099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 799/1570] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:29:43.241892 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5fdb-3cae-4205-b40c-3fbe836d0f0e admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.246354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:43.251271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a5d0-4ffa-41fb-9a79-dc9778abaa30 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.259837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd8b0d-a436-4ebb-8a58-dfdf72b50972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.260515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd8b0d-a436-4ebb-8a58-dfdf72b50972 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.261061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd8b0d-a436-4ebb-8a58-dfdf72b50972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.261348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd8b0d-a436-4ebb-8a58-dfdf72b50972 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.261795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5fdb-3cae-4205-b40c-3fbe836d0f0e admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.269568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd8b0d-a436-4ebb-8a58-dfdf72b50972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.269568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd8b0d-a436-4ebb-8a58-dfdf72b50972 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.269810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3e62d-b736-4836-a7fd-9155a776164b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.269810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3e62d-b736-4836-a7fd-9155a776164b admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.269810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3e62d-b736-4836-a7fd-9155a776164b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.269810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3e62d-b736-4836-a7fd-9155a776164b admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.270401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3e62d-b736-4836-a7fd-9155a776164b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.272321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3e62d-b736-4836-a7fd-9155a776164b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.295619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 772/1571] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/users => generated 290 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.296756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd8b0d-a436-4ebb-8a58-dfdf72b50972 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.306330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 800/1572] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.307718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3e62d-b736-4836-a7fd-9155a776164b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.311632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd8b0d-a436-4ebb-8a58-dfdf72b50972 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.370964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3e62d-b736-4836-a7fd-9155a776164b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.372550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7ae9c-7dd6-405c-b7aa-7ef9a8385fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.374143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7ae9c-7dd6-405c-b7aa-7ef9a8385fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.374428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7ae9c-7dd6-405c-b7aa-7ef9a8385fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.374708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7ae9c-7dd6-405c-b7aa-7ef9a8385fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.375396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7ae9c-7dd6-405c-b7aa-7ef9a8385fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.376278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adaab70-b94f-4653-aeab-06f797624477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.376998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adaab70-b94f-4653-aeab-06f797624477 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.377283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adaab70-b94f-4653-aeab-06f797624477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.377413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7ae9c-7dd6-405c-b7aa-7ef9a8385fc7 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.377557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adaab70-b94f-4653-aeab-06f797624477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.378273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adaab70-b94f-4653-aeab-06f797624477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.379831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adaab70-b94f-4653-aeab-06f797624477 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.390979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 801/1573] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.391764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adaab70-b94f-4653-aeab-06f797624477 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.401507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 773/1574] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.401507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7ae9c-7dd6-405c-b7aa-7ef9a8385fc7 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.405048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adaab70-b94f-4653-aeab-06f797624477 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.410179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7ae9c-7dd6-405c-b7aa-7ef9a8385fc7 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.410346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ad135-79a7-4b3d-8b15-7083dd95c54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.410999 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ad135-79a7-4b3d-8b15-7083dd95c54c admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.411292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ad135-79a7-4b3d-8b15-7083dd95c54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.411659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ad135-79a7-4b3d-8b15-7083dd95c54c admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.412357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ad135-79a7-4b3d-8b15-7083dd95c54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.414452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ad135-79a7-4b3d-8b15-7083dd95c54c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.414831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6935f-9462-40d2-9278-7a75efcea5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.415475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6935f-9462-40d2-9278-7a75efcea5bf admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.415751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6935f-9462-40d2-9278-7a75efcea5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.416022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6935f-9462-40d2-9278-7a75efcea5bf admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.416689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6935f-9462-40d2-9278-7a75efcea5bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.418631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6935f-9462-40d2-9278-7a75efcea5bf admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.445937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 774/1575] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/users => generated 284 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.446141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 802/1576] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.446258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6935f-9462-40d2-9278-7a75efcea5bf admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.446340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ad135-79a7-4b3d-8b15-7083dd95c54c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.456020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6935f-9462-40d2-9278-7a75efcea5bf admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.506245 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ad135-79a7-4b3d-8b15-7083dd95c54c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.511437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8c589-30d5-422d-86c3-651fd8571857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.512095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8c589-30d5-422d-86c3-651fd8571857 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.512380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8c589-30d5-422d-86c3-651fd8571857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.512645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8c589-30d5-422d-86c3-651fd8571857 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.513386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8c589-30d5-422d-86c3-651fd8571857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.514489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80fb9f-b493-49c0-ae78-ab0af78b3021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.515220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80fb9f-b493-49c0-ae78-ab0af78b3021 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.515511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80fb9f-b493-49c0-ae78-ab0af78b3021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.515795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80fb9f-b493-49c0-ae78-ab0af78b3021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.515971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8c589-30d5-422d-86c3-651fd8571857 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.516557 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80fb9f-b493-49c0-ae78-ab0af78b3021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.518402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80fb9f-b493-49c0-ae78-ab0af78b3021 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.532075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 775/1577] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.532075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80fb9f-b493-49c0-ae78-ab0af78b3021 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.542186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80fb9f-b493-49c0-ae78-ab0af78b3021 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.548961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 803/1578] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.548961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8c589-30d5-422d-86c3-651fd8571857 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.550776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27281-2c89-4362-bfe4-fd7569458637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.551440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27281-2c89-4362-bfe4-fd7569458637 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.551724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27281-2c89-4362-bfe4-fd7569458637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.551998 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27281-2c89-4362-bfe4-fd7569458637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.552759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27281-2c89-4362-bfe4-fd7569458637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.554416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27281-2c89-4362-bfe4-fd7569458637 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.565429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 776/1579] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.568528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27281-2c89-4362-bfe4-fd7569458637 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.577339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27281-2c89-4362-bfe4-fd7569458637 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.578286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:43.612025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8c589-30d5-422d-86c3-651fd8571857 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.616989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b539bc-b637-4d9d-8319-1771649ac2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.617681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b539bc-b637-4d9d-8319-1771649ac2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.618012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b539bc-b637-4d9d-8319-1771649ac2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.618238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b539bc-b637-4d9d-8319-1771649ac2a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.619242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b539bc-b637-4d9d-8319-1771649ac2a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.622127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b539bc-b637-4d9d-8319-1771649ac2a5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.636209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:43.644180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 804/1580] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.645038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b539bc-b637-4d9d-8319-1771649ac2a5 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.655072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b539bc-b637-4d9d-8319-1771649ac2a5 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.656974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731a330-93a2-4235-8aa0-6d12775dfd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.657651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731a330-93a2-4235-8aa0-6d12775dfd91 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.657936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731a330-93a2-4235-8aa0-6d12775dfd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.658282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731a330-93a2-4235-8aa0-6d12775dfd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.658943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731a330-93a2-4235-8aa0-6d12775dfd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.659831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa2a61-8341-40a8-a045-13c091cbaa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.660503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa2a61-8341-40a8-a045-13c091cbaa53 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.660621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731a330-93a2-4235-8aa0-6d12775dfd91 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.660786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa2a61-8341-40a8-a045-13c091cbaa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.661093 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa2a61-8341-40a8-a045-13c091cbaa53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.661839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa2a61-8341-40a8-a045-13c091cbaa53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.663409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa2a61-8341-40a8-a045-13c091cbaa53 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.670676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 777/1581] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.671507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731a330-93a2-4235-8aa0-6d12775dfd91 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.676932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 805/1582] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.676932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa2a61-8341-40a8-a045-13c091cbaa53 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.681304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731a330-93a2-4235-8aa0-6d12775dfd91 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.686187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9eda1c-a5e4-47ad-a67c-394576e59b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.686379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa2a61-8341-40a8-a045-13c091cbaa53 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.686916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eda1c-a5e4-47ad-a67c-394576e59b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.687134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eda1c-a5e4-47ad-a67c-394576e59b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.687410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eda1c-a5e4-47ad-a67c-394576e59b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.688147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9eda1c-a5e4-47ad-a67c-394576e59b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.689778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9eda1c-a5e4-47ad-a67c-394576e59b4b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.691301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd17e-3c75-4a5f-9342-69207b82f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.691973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd17e-3c75-4a5f-9342-69207b82f84b admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.692260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd17e-3c75-4a5f-9342-69207b82f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.692540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd17e-3c75-4a5f-9342-69207b82f84b admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.693280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebd17e-3c75-4a5f-9342-69207b82f84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.695232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebd17e-3c75-4a5f-9342-69207b82f84b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.703493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 778/1583] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.703493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9eda1c-a5e4-47ad-a67c-394576e59b4b admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.711702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9eda1c-a5e4-47ad-a67c-394576e59b4b admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.716364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b86a24-6ffd-4595-a3d7-c717e5965c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.719912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 806/1584] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.720063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b86a24-6ffd-4595-a3d7-c717e5965c6e admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.720063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b86a24-6ffd-4595-a3d7-c717e5965c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.720222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd17e-3c75-4a5f-9342-69207b82f84b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.720300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b86a24-6ffd-4595-a3d7-c717e5965c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.720300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b86a24-6ffd-4595-a3d7-c717e5965c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.720300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b86a24-6ffd-4595-a3d7-c717e5965c6e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.729603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd17e-3c75-4a5f-9342-69207b82f84b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.735186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 779/1585] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.735186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b86a24-6ffd-4595-a3d7-c717e5965c6e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.735424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29553a3b-b701-496b-895b-56b51a322e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.735627 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29553a3b-b701-496b-895b-56b51a322e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.735914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29553a3b-b701-496b-895b-56b51a322e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.736194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29553a3b-b701-496b-895b-56b51a322e99 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.736907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29553a3b-b701-496b-895b-56b51a322e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.738838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29553a3b-b701-496b-895b-56b51a322e99 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.743504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b86a24-6ffd-4595-a3d7-c717e5965c6e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.748838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a198b5-81da-4aec-a050-1b2febd5896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.749518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a198b5-81da-4aec-a050-1b2febd5896e admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.749800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a198b5-81da-4aec-a050-1b2febd5896e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.750082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a198b5-81da-4aec-a050-1b2febd5896e admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.751058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a198b5-81da-4aec-a050-1b2febd5896e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.752859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a198b5-81da-4aec-a050-1b2febd5896e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.759399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 807/1586] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:43 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:43.760225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29553a3b-b701-496b-895b-56b51a322e99 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.769734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 780/1587] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.769734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a198b5-81da-4aec-a050-1b2febd5896e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.771611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29553a3b-b701-496b-895b-56b51a322e99 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.776344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a6c30-113f-4e15-a7bb-4ed83895d998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.777043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a6c30-113f-4e15-a7bb-4ed83895d998 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.777229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a198b5-81da-4aec-a050-1b2febd5896e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.777334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a6c30-113f-4e15-a7bb-4ed83895d998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.777590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a6c30-113f-4e15-a7bb-4ed83895d998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.778319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a6c30-113f-4e15-a7bb-4ed83895d998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.780559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a6c30-113f-4e15-a7bb-4ed83895d998 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.782666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2101f2-538c-43c4-9236-c34e9b0d5be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.783559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2101f2-538c-43c4-9236-c34e9b0d5be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.783837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2101f2-538c-43c4-9236-c34e9b0d5be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.784111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2101f2-538c-43c4-9236-c34e9b0d5be4 admin admin] PATH_INFO: `/v3/projects/65c6b1563c424330ade1ef15f8efa110/users/139e3e42acff4c4ca8df0a7c4fc81053/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.791327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 808/1588] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.792149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a6c30-113f-4e15-a7bb-4ed83895d998 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.799713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2101f2-538c-43c4-9236-c34e9b0d5be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c6b1563c424330ade1ef15f8efa110, user_id=139e3e42acff4c4ca8df0a7c4fc81053, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.802350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2101f2-538c-43c4-9236-c34e9b0d5be4 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.802508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a6c30-113f-4e15-a7bb-4ed83895d998 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.807825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca176ef-d483-4805-b9db-61a7357b36c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.808632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca176ef-d483-4805-b9db-61a7357b36c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.808940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca176ef-d483-4805-b9db-61a7357b36c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.809226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca176ef-d483-4805-b9db-61a7357b36c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.810196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca176ef-d483-4805-b9db-61a7357b36c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.812204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca176ef-d483-4805-b9db-61a7357b36c9 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.817730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 781/1589] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:43 2022] PUT /identity/v3/projects/65c6b1563c424330ade1ef15f8efa110/users/139e3e42acff4c4ca8df0a7c4fc81053/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:43.819111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2101f2-538c-43c4-9236-c34e9b0d5be4 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.824107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 809/1590] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.824981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca176ef-d483-4805-b9db-61a7357b36c9 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.834738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca176ef-d483-4805-b9db-61a7357b36c9 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.883513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2101f2-538c-43c4-9236-c34e9b0d5be4 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.888438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d09d7c-97b0-48c6-ae75-528b96a87172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.889832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d09d7c-97b0-48c6-ae75-528b96a87172 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.890151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d09d7c-97b0-48c6-ae75-528b96a87172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.890430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d09d7c-97b0-48c6-ae75-528b96a87172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.891162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d09d7c-97b0-48c6-ae75-528b96a87172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.891770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a571f-c064-4c1c-a96a-57a297b8be9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.892446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a571f-c064-4c1c-a96a-57a297b8be9d admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.892761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a571f-c064-4c1c-a96a-57a297b8be9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.892858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d09d7c-97b0-48c6-ae75-528b96a87172 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.893075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a571f-c064-4c1c-a96a-57a297b8be9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.894037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a571f-c064-4c1c-a96a-57a297b8be9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.895646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a571f-c064-4c1c-a96a-57a297b8be9d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.902880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 782/1591] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.903514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d09d7c-97b0-48c6-ae75-528b96a87172 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.908740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 810/1592] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.908740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a571f-c064-4c1c-a96a-57a297b8be9d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.913902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d09d7c-97b0-48c6-ae75-528b96a87172 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.916256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a571f-c064-4c1c-a96a-57a297b8be9d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.918438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ef74c-5982-4e79-be1c-97be3b9064ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.919302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ef74c-5982-4e79-be1c-97be3b9064ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.919586 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ef74c-5982-4e79-be1c-97be3b9064ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.919944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ef74c-5982-4e79-be1c-97be3b9064ed admin admin] PATH_INFO: `/v3/projects/65624b5cd9aa4aeead4174807d1be2e3/users/058ee448f819446f9ff20d57b12b3b3a/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.921227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6796e5-f3e7-4537-ba3d-028102d93279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:43.921875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6796e5-f3e7-4537-ba3d-028102d93279 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:43.922197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6796e5-f3e7-4537-ba3d-028102d93279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:43.922510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6796e5-f3e7-4537-ba3d-028102d93279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:43.923249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6796e5-f3e7-4537-ba3d-028102d93279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.925072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6796e5-f3e7-4537-ba3d-028102d93279 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.934454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ef74c-5982-4e79-be1c-97be3b9064ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=65624b5cd9aa4aeead4174807d1be2e3, user_id=058ee448f819446f9ff20d57b12b3b3a, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:43.936334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 811/1593] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:43.937215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6796e5-f3e7-4537-ba3d-028102d93279 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.938163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ef74c-5982-4e79-be1c-97be3b9064ed admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:43.949260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6796e5-f3e7-4537-ba3d-028102d93279 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:43.953065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 783/1594] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:43 2022] PUT /identity/v3/projects/65624b5cd9aa4aeead4174807d1be2e3/users/058ee448f819446f9ff20d57b12b3b3a/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:43.953884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ef74c-5982-4e79-be1c-97be3b9064ed admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:43.985212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:44.011254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c712372-fb41-48a4-8d45-61f88f841aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.012163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c712372-fb41-48a4-8d45-61f88f841aff admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.012461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c712372-fb41-48a4-8d45-61f88f841aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.012767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c712372-fb41-48a4-8d45-61f88f841aff admin admin] PATH_INFO: `/v3/projects/377219e53ce94ab089e48437d3468663/users/e91e38a712074ca5a2567da012445aa3/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.013011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:44.019760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ef74c-5982-4e79-be1c-97be3b9064ed admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.024893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bfd0e-d74f-4aed-bcf0-5692fdc9fc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.025569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bfd0e-d74f-4aed-bcf0-5692fdc9fc04 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.025950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bfd0e-d74f-4aed-bcf0-5692fdc9fc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.026157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bfd0e-d74f-4aed-bcf0-5692fdc9fc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.026842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bfd0e-d74f-4aed-bcf0-5692fdc9fc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.027557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c712372-fb41-48a4-8d45-61f88f841aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=377219e53ce94ab089e48437d3468663, user_id=e91e38a712074ca5a2567da012445aa3, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.028519 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bfd0e-d74f-4aed-bcf0-5692fdc9fc04 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.028625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING py.warnings [None req-3c712372-fb41-48a4-8d45-61f88f841aff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:29:44.028625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: warnings.warn(msg) May 23 15:29:44.028625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: May 23 15:29:44.031221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c712372-fb41-48a4-8d45-61f88f841aff admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.040542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 784/1595] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:44.041941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a96670-647b-4743-b726-1c01e769b224 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.042236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a96670-647b-4743-b726-1c01e769b224 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.042520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a96670-647b-4743-b726-1c01e769b224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.043976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 812/1596] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:43 2022] PUT /identity/v3/projects/377219e53ce94ab089e48437d3468663/users/e91e38a712074ca5a2567da012445aa3/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:44.045233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c712372-fb41-48a4-8d45-61f88f841aff admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.064383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-79a96670-647b-4743-b726-1c01e769b224 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:44.065563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-79a96670-647b-4743-b726-1c01e769b224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:44.104762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c712372-fb41-48a4-8d45-61f88f841aff admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.109877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ee7a1-557d-408c-bafa-118d326b2f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.110807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ee7a1-557d-408c-bafa-118d326b2f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.111075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ee7a1-557d-408c-bafa-118d326b2f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.111351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ee7a1-557d-408c-bafa-118d326b2f4b admin admin] PATH_INFO: `/v3/projects/f2020e2bf35744af9161d7cf3e8bd345/users/06431315dc8040cbbafbe83a3ed8e0e4/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.124180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 785/1597] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:44.125120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a96670-647b-4743-b726-1c01e769b224 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.126583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ee7a1-557d-408c-bafa-118d326b2f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2020e2bf35744af9161d7cf3e8bd345, user_id=06431315dc8040cbbafbe83a3ed8e0e4, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.129378 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ee7a1-557d-408c-bafa-118d326b2f4b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.135358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a96670-647b-4743-b726-1c01e769b224 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.140938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42abb7b-d97b-48c6-b5c5-39a9b950e90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.141846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42abb7b-d97b-48c6-b5c5-39a9b950e90d admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.142127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42abb7b-d97b-48c6-b5c5-39a9b950e90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.142402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42abb7b-d97b-48c6-b5c5-39a9b950e90d admin admin] PATH_INFO: `/v3/projects/1aebe2f4fe924c0093c13e4f92070f0b/users/8776d23acd7048faabbafa37a3028dd6/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.143716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 813/1598] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:44 2022] PUT /identity/v3/projects/f2020e2bf35744af9161d7cf3e8bd345/users/06431315dc8040cbbafbe83a3ed8e0e4/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:44.144563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ee7a1-557d-408c-bafa-118d326b2f4b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.161936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42abb7b-d97b-48c6-b5c5-39a9b950e90d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aebe2f4fe924c0093c13e4f92070f0b, user_id=8776d23acd7048faabbafa37a3028dd6, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.164653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42abb7b-d97b-48c6-b5c5-39a9b950e90d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.179015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 786/1599] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:44 2022] PUT /identity/v3/projects/1aebe2f4fe924c0093c13e4f92070f0b/users/8776d23acd7048faabbafa37a3028dd6/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:44.179903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42abb7b-d97b-48c6-b5c5-39a9b950e90d admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.217935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ee7a1-557d-408c-bafa-118d326b2f4b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.245439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42abb7b-d97b-48c6-b5c5-39a9b950e90d admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.250899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb2658-0e3b-432a-92eb-2f348dd7d169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.251572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb2658-0e3b-432a-92eb-2f348dd7d169 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.251858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb2658-0e3b-432a-92eb-2f348dd7d169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.252136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb2658-0e3b-432a-92eb-2f348dd7d169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.253016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb2658-0e3b-432a-92eb-2f348dd7d169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.254640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb2658-0e3b-432a-92eb-2f348dd7d169 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.264896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 787/1600] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:44.266225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083d490-0781-4322-af34-bd52bd3f6fa4 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.266518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083d490-0781-4322-af34-bd52bd3f6fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.266796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083d490-0781-4322-af34-bd52bd3f6fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.269443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:44.282544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec0da9-0ae1-4386-bd4a-65e8f1f4bbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.283221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec0da9-0ae1-4386-bd4a-65e8f1f4bbdd admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.283502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec0da9-0ae1-4386-bd4a-65e8f1f4bbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.283778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec0da9-0ae1-4386-bd4a-65e8f1f4bbdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.284531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec0da9-0ae1-4386-bd4a-65e8f1f4bbdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.286217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec0da9-0ae1-4386-bd4a-65e8f1f4bbdd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.295593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d083d490-0781-4322-af34-bd52bd3f6fa4 None None] MFA Rules not processed for user `058ee448f819446f9ff20d57b12b3b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:44.297029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d083d490-0781-4322-af34-bd52bd3f6fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:44.297582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 814/1601] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:44.297846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec0da9-0ae1-4386-bd4a-65e8f1f4bbdd admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.307493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec0da9-0ae1-4386-bd4a-65e8f1f4bbdd admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.313086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d77bf-da3e-4799-8ef1-f2da2ac759c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.313973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d77bf-da3e-4799-8ef1-f2da2ac759c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.314253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d77bf-da3e-4799-8ef1-f2da2ac759c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.314583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d77bf-da3e-4799-8ef1-f2da2ac759c5 admin admin] PATH_INFO: `/v3/projects/9aaff9db173d4618929536b08d2be8f0/users/d0ae34a029c14c6f826456414ac0670d/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.330006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d77bf-da3e-4799-8ef1-f2da2ac759c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aaff9db173d4618929536b08d2be8f0, user_id=d0ae34a029c14c6f826456414ac0670d, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.332673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d77bf-da3e-4799-8ef1-f2da2ac759c5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.345748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 815/1602] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:44 2022] PUT /identity/v3/projects/9aaff9db173d4618929536b08d2be8f0/users/d0ae34a029c14c6f826456414ac0670d/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:44.347151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49293fd-31f3-498c-9eca-119163232645 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.347458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49293fd-31f3-498c-9eca-119163232645 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.347782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49293fd-31f3-498c-9eca-119163232645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.356335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:44.364654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b49293fd-31f3-498c-9eca-119163232645 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:44.365898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b49293fd-31f3-498c-9eca-119163232645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:44.367864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 788/1603] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:44.368779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083d490-0781-4322-af34-bd52bd3f6fa4 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.424844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083d490-0781-4322-af34-bd52bd3f6fa4 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.426536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 816/1604] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:44.427884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb3f72-d440-46a7-97e0-0e5358433c86 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.428243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb3f72-d440-46a7-97e0-0e5358433c86 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.428590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb3f72-d440-46a7-97e0-0e5358433c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.446417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b1bb3f72-d440-46a7-97e0-0e5358433c86 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:44.447605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b1bb3f72-d440-46a7-97e0-0e5358433c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:44.458270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 817/1605] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:44.459533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918b754-ef10-47ed-a6b1-883c4160e8f5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.459821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918b754-ef10-47ed-a6b1-883c4160e8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.460092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918b754-ef10-47ed-a6b1-883c4160e8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.485203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7bfbf-4254-4fae-ae1a-44ff2b342ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.486011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7bfbf-4254-4fae-ae1a-44ff2b342ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.486303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7bfbf-4254-4fae-ae1a-44ff2b342ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.486580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7bfbf-4254-4fae-ae1a-44ff2b342ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.486992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f918b754-ef10-47ed-a6b1-883c4160e8f5 None None] MFA Rules not processed for user `8776d23acd7048faabbafa37a3028dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:44.488173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f918b754-ef10-47ed-a6b1-883c4160e8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:44.495695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7bfbf-4254-4fae-ae1a-44ff2b342ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.497238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7bfbf-4254-4fae-ae1a-44ff2b342ae4 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.507783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 789/1606] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:44.508605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7bfbf-4254-4fae-ae1a-44ff2b342ae4 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.519988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7bfbf-4254-4fae-ae1a-44ff2b342ae4 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.525737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4c359-5582-4c61-b681-67d389aa426a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.526492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4c359-5582-4c61-b681-67d389aa426a admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.526773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4c359-5582-4c61-b681-67d389aa426a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.527047 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4c359-5582-4c61-b681-67d389aa426a admin admin] PATH_INFO: `/v3/projects/da1811a65421483f9fed0285deca97a2/users/bd79f566bbcd4725b358eea50996447a/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.544337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4c359-5582-4c61-b681-67d389aa426a admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1811a65421483f9fed0285deca97a2, user_id=bd79f566bbcd4725b358eea50996447a, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.547832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4c359-5582-4c61-b681-67d389aa426a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.560547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 818/1607] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:44.561436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918b754-ef10-47ed-a6b1-883c4160e8f5 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.562774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 790/1608] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:44 2022] PUT /identity/v3/projects/da1811a65421483f9fed0285deca97a2/users/bd79f566bbcd4725b358eea50996447a/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:44.564295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4d499-aa0c-463a-a723-84f6a447aa1a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.564642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4d499-aa0c-463a-a723-84f6a447aa1a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.564931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4d499-aa0c-463a-a723-84f6a447aa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.583352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918b754-ef10-47ed-a6b1-883c4160e8f5 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.590296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b1e4d499-aa0c-463a-a723-84f6a447aa1a None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:44.592302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b1e4d499-aa0c-463a-a723-84f6a447aa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:44.657202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:44.678082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5ee5c-ec2f-49cf-adbc-7fa08fc43a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.678993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5ee5c-ec2f-49cf-adbc-7fa08fc43a89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.679283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5ee5c-ec2f-49cf-adbc-7fa08fc43a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.679565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5ee5c-ec2f-49cf-adbc-7fa08fc43a89 admin admin] PATH_INFO: `/v3/projects/0712b807a4584f7b8f019ceb65ff1e14/users/5074b9fa045543b9a62bf751e736d0df/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.681772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 791/1609] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:44.682649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4d499-aa0c-463a-a723-84f6a447aa1a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.693455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4d499-aa0c-463a-a723-84f6a447aa1a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.698788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d69b34-50c7-4630-a189-9c43cd3cecc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.699559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d69b34-50c7-4630-a189-9c43cd3cecc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.700072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d69b34-50c7-4630-a189-9c43cd3cecc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.700456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d69b34-50c7-4630-a189-9c43cd3cecc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.701525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d69b34-50c7-4630-a189-9c43cd3cecc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.702365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5ee5c-ec2f-49cf-adbc-7fa08fc43a89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0712b807a4584f7b8f019ceb65ff1e14, user_id=5074b9fa045543b9a62bf751e736d0df, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.704218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d69b34-50c7-4630-a189-9c43cd3cecc4 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.705259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5ee5c-ec2f-49cf-adbc-7fa08fc43a89 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.721359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 819/1610] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:44 2022] PUT /identity/v3/projects/0712b807a4584f7b8f019ceb65ff1e14/users/5074b9fa045543b9a62bf751e736d0df/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:44.722306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5ee5c-ec2f-49cf-adbc-7fa08fc43a89 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.734596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 792/1611] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:44 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:44.735359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d69b34-50c7-4630-a189-9c43cd3cecc4 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.737140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c1d69b34-50c7-4630-a189-9c43cd3cecc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:44.747399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:44.793342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5ee5c-ec2f-49cf-adbc-7fa08fc43a89 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.814141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d69b34-50c7-4630-a189-9c43cd3cecc4 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.818223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1ec3f-f9f8-4f19-810a-f8792a90c388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.818896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1ec3f-f9f8-4f19-810a-f8792a90c388 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.819259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1ec3f-f9f8-4f19-810a-f8792a90c388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.819631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1ec3f-f9f8-4f19-810a-f8792a90c388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.866593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d614f5-195c-4868-90e8-6e0306fa5b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.867269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d614f5-195c-4868-90e8-6e0306fa5b2f service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.867552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d614f5-195c-4868-90e8-6e0306fa5b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.867858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d614f5-195c-4868-90e8-6e0306fa5b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.896520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1ec3f-f9f8-4f19-810a-f8792a90c388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.898843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1ec3f-f9f8-4f19-810a-f8792a90c388 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.911194 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 793/1612] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:44.912083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1ec3f-f9f8-4f19-810a-f8792a90c388 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:44.941725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d614f5-195c-4868-90e8-6e0306fa5b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:44.943408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d614f5-195c-4868-90e8-6e0306fa5b2f service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:44.953943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 820/1613] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:44.955091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44997db2-3ca6-4af0-80b0-1dafc07f4437 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.955506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44997db2-3ca6-4af0-80b0-1dafc07f4437 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.955794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44997db2-3ca6-4af0-80b0-1dafc07f4437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:44.984196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1ec3f-f9f8-4f19-810a-f8792a90c388 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:44.989045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-44997db2-3ca6-4af0-80b0-1dafc07f4437 None None] MFA Rules not processed for user `bd79f566bbcd4725b358eea50996447a`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:44.989184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd684ab-e2a2-4551-9a2a-d64d3ee47b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:44.990443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-44997db2-3ca6-4af0-80b0-1dafc07f4437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:44.990778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd684ab-e2a2-4551-9a2a-d64d3ee47b75 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:44.991106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd684ab-e2a2-4551-9a2a-d64d3ee47b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:44.991396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd684ab-e2a2-4551-9a2a-d64d3ee47b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.020547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:45.068025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd684ab-e2a2-4551-9a2a-d64d3ee47b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.070080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd684ab-e2a2-4551-9a2a-d64d3ee47b75 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.088516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 794/1614] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:45.089472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd684ab-e2a2-4551-9a2a-d64d3ee47b75 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:45.100416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 821/1615] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:45.102886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f2013-0724-46e1-aa80-edf91f595f0f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.103472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f2013-0724-46e1-aa80-edf91f595f0f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.103727 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f2013-0724-46e1-aa80-edf91f595f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.104007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd684ab-e2a2-4551-9a2a-d64d3ee47b75 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:45.109673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf68e8-500a-43e3-9799-23e8954aa3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:45.110449 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf68e8-500a-43e3-9799-23e8954aa3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.110797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf68e8-500a-43e3-9799-23e8954aa3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.111163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf68e8-500a-43e3-9799-23e8954aa3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.124673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf68e8-500a-43e3-9799-23e8954aa3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.126264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf68e8-500a-43e3-9799-23e8954aa3d8 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.131626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:45.138943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a14f2013-0724-46e1-aa80-edf91f595f0f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:45.140331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a14f2013-0724-46e1-aa80-edf91f595f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:45.142103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 795/1616] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:45.143274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf68e8-500a-43e3-9799-23e8954aa3d8 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:45.154427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 822/1617] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:45.156579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205f1ca-2575-48f8-a897-ff26b599bd8e None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.157054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205f1ca-2575-48f8-a897-ff26b599bd8e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.157550 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205f1ca-2575-48f8-a897-ff26b599bd8e None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.158419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf68e8-500a-43e3-9799-23e8954aa3d8 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:45.159701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 823/1618] 10.222.0.30 () {54 vars in 961 bytes} [Mon May 23 15:29:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:29:45.164346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfa26b-55a9-4687-b4bd-29c22c56ef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:45.165093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfa26b-55a9-4687-b4bd-29c22c56ef45 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.165437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfa26b-55a9-4687-b4bd-29c22c56ef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.166016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfa26b-55a9-4687-b4bd-29c22c56ef45 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.166944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfa26b-55a9-4687-b4bd-29c22c56ef45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.170512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfa26b-55a9-4687-b4bd-29c22c56ef45 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.180361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4086c-b887-48ab-8f43-e7afa8cd9d1e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.181522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4086c-b887-48ab-8f43-e7afa8cd9d1e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.182201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4086c-b887-48ab-8f43-e7afa8cd9d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.200922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 796/1619] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:45 2022] POST /identity/v3/users => generated 291 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:45.200922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfa26b-55a9-4687-b4bd-29c22c56ef45 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:45.211570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a4f4086c-b887-48ab-8f43-e7afa8cd9d1e None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:45.213085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a4f4086c-b887-48ab-8f43-e7afa8cd9d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:45.218645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfa26b-55a9-4687-b4bd-29c22c56ef45 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:45.224983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a568e-0dee-4c7c-8ea8-2f7b9a0919a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:45.227128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a568e-0dee-4c7c-8ea8-2f7b9a0919a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.227569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a568e-0dee-4c7c-8ea8-2f7b9a0919a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.228068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a568e-0dee-4c7c-8ea8-2f7b9a0919a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.248796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a568e-0dee-4c7c-8ea8-2f7b9a0919a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.251260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a568e-0dee-4c7c-8ea8-2f7b9a0919a6 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.288289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 797/1620] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:45.290805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a568e-0dee-4c7c-8ea8-2f7b9a0919a6 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:45.313476 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a568e-0dee-4c7c-8ea8-2f7b9a0919a6 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:45.317615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 824/1621] 10.222.0.30 () {58 vars in 1058 bytes} [Mon May 23 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:29:45.318897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4086c-b887-48ab-8f43-e7afa8cd9d1e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:45.324374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81643f69-e401-488e-8a4d-19d77cfe2b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:45.326969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81643f69-e401-488e-8a4d-19d77cfe2b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.327742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81643f69-e401-488e-8a4d-19d77cfe2b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.329072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81643f69-e401-488e-8a4d-19d77cfe2b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.330251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81643f69-e401-488e-8a4d-19d77cfe2b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.333383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81643f69-e401-488e-8a4d-19d77cfe2b03 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.334982 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4086c-b887-48ab-8f43-e7afa8cd9d1e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:45.354063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 798/1622] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:45.360346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1bd0b-3394-461e-a0a0-f26407d3558f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:45.361154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1bd0b-3394-461e-a0a0-f26407d3558f service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.361517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1bd0b-3394-461e-a0a0-f26407d3558f service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.361891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1bd0b-3394-461e-a0a0-f26407d3558f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.363241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81643f69-e401-488e-8a4d-19d77cfe2b03 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:45.382146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1bd0b-3394-461e-a0a0-f26407d3558f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.383114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81643f69-e401-488e-8a4d-19d77cfe2b03 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:45.383785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1bd0b-3394-461e-a0a0-f26407d3558f service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.388746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae73ad2-69a0-4e00-a0f3-8700527633e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:45.389833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae73ad2-69a0-4e00-a0f3-8700527633e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.390125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae73ad2-69a0-4e00-a0f3-8700527633e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.390407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae73ad2-69a0-4e00-a0f3-8700527633e8 admin admin] PATH_INFO: `/v3/projects/9ebc5b8a5fc64b6aa20fd12a5cc8899b/users/c3682c6c68b24c4caaaa5b81f63dc477/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.400421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 825/1623] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:29:45.417092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae73ad2-69a0-4e00-a0f3-8700527633e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebc5b8a5fc64b6aa20fd12a5cc8899b, user_id=c3682c6c68b24c4caaaa5b81f63dc477, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.420787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae73ad2-69a0-4e00-a0f3-8700527633e8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.445339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 799/1624] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:45 2022] PUT /identity/v3/projects/9ebc5b8a5fc64b6aa20fd12a5cc8899b/users/c3682c6c68b24c4caaaa5b81f63dc477/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:45.446602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1bd0b-3394-461e-a0a0-f26407d3558f service nova] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:45.458528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1bd0b-3394-461e-a0a0-f26407d3558f service nova] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:45.533282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62292cda-54ec-49a9-a695-914586d8075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:45.534362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62292cda-54ec-49a9-a695-914586d8075c admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.535124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62292cda-54ec-49a9-a695-914586d8075c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.535716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62292cda-54ec-49a9-a695-914586d8075c admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.536717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62292cda-54ec-49a9-a695-914586d8075c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.538830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62292cda-54ec-49a9-a695-914586d8075c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.568623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 826/1625] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:45.570411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae73ad2-69a0-4e00-a0f3-8700527633e8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:45.597998 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae73ad2-69a0-4e00-a0f3-8700527633e8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:45.611073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296c32f-9d33-40e3-af93-879b08860d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:45.613951 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296c32f-9d33-40e3-af93-879b08860d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.614560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296c32f-9d33-40e3-af93-879b08860d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.614972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296c32f-9d33-40e3-af93-879b08860d63 admin admin] PATH_INFO: `/v3/projects/9ebc5b8a5fc64b6aa20fd12a5cc8899b/users/c3682c6c68b24c4caaaa5b81f63dc477/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.617627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296c32f-9d33-40e3-af93-879b08860d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebc5b8a5fc64b6aa20fd12a5cc8899b, user_id=c3682c6c68b24c4caaaa5b81f63dc477, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.623421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296c32f-9d33-40e3-af93-879b08860d63 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.667313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 800/1626] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:45 2022] PUT /identity/v3/projects/9ebc5b8a5fc64b6aa20fd12a5cc8899b/users/c3682c6c68b24c4caaaa5b81f63dc477/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:45.670811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4abb7-b5b6-434b-bd32-d459d73102e5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.671436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4abb7-b5b6-434b-bd32-d459d73102e5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.671883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4abb7-b5b6-434b-bd32-d459d73102e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.710155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f0d4abb7-b5b6-434b-bd32-d459d73102e5 None None] MFA Rules not processed for user `c3682c6c68b24c4caaaa5b81f63dc477`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:45.711419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f0d4abb7-b5b6-434b-bd32-d459d73102e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:45.735756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:45.822651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 827/1627] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:45.834372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296c32f-9d33-40e3-af93-879b08860d63 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:45.929163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296c32f-9d33-40e3-af93-879b08860d63 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:45.939250 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe062a55-d4b8-4b0c-acce-186fb52a1ba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:45.943209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062a55-d4b8-4b0c-acce-186fb52a1ba9 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:45.943209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062a55-d4b8-4b0c-acce-186fb52a1ba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:45.943209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062a55-d4b8-4b0c-acce-186fb52a1ba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:45.972277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe062a55-d4b8-4b0c-acce-186fb52a1ba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:45.974292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe062a55-d4b8-4b0c-acce-186fb52a1ba9 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:45.995981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 801/1628] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:46.409485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d61ad-1dd2-4e2e-9fe3-a584aa14a65c None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:46.410074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d61ad-1dd2-4e2e-9fe3-a584aa14a65c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:46.410074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d61ad-1dd2-4e2e-9fe3-a584aa14a65c None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:46.412089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 828/1629] 10.222.0.30 () {54 vars in 961 bytes} [Mon May 23 15:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:29:46.442569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a03b3-b347-4eab-be44-293f3e12ecd6 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:46.444003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a03b3-b347-4eab-be44-293f3e12ecd6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:46.445046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a03b3-b347-4eab-be44-293f3e12ecd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:46.495156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-fe5a03b3-b347-4eab-be44-293f3e12ecd6 None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:46.501110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-fe5a03b3-b347-4eab-be44-293f3e12ecd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:46.707096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 802/1630] 10.222.0.30 () {58 vars in 1058 bytes} [Mon May 23 15:29:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:46.714182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce95a66-513c-4349-a5a8-793b29a9c74e None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:46.714465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce95a66-513c-4349-a5a8-793b29a9c74e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:46.714749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce95a66-513c-4349-a5a8-793b29a9c74e None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:46.716642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 829/1631] 10.222.0.30 () {54 vars in 912 bytes} [Mon May 23 15:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:29:46.723579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad81e0-6598-4ecb-afc9-60c4f904952b None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:46.723863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad81e0-6598-4ecb-afc9-60c4f904952b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:46.724144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad81e0-6598-4ecb-afc9-60c4f904952b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:46.770364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b6ad81e0-6598-4ecb-afc9-60c4f904952b None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:46.771673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b6ad81e0-6598-4ecb-afc9-60c4f904952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:46.800536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 803/1632] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:29:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:46.822097 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce95a66-513c-4349-a5a8-793b29a9c74e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:46.843777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce95a66-513c-4349-a5a8-793b29a9c74e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:46.847645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a637d5-8746-4080-b670-938803c54233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:46.848300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a637d5-8746-4080-b670-938803c54233 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:46.848594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a637d5-8746-4080-b670-938803c54233 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:46.848854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a637d5-8746-4080-b670-938803c54233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:46.860466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a637d5-8746-4080-b670-938803c54233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:46.862134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a637d5-8746-4080-b670-938803c54233 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:46.878798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 830/1633] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:29:47.031146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de90d3-b70e-4811-a01b-7c35f8b1394d None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:47.032210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de90d3-b70e-4811-a01b-7c35f8b1394d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:47.032491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de90d3-b70e-4811-a01b-7c35f8b1394d None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:47.034156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 804/1634] 10.222.0.30 () {54 vars in 912 bytes} [Mon May 23 15:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:29:47.045229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261edf8-1bfa-481f-965c-6094da3c4639 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:47.045681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261edf8-1bfa-481f-965c-6094da3c4639 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:47.046102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261edf8-1bfa-481f-965c-6094da3c4639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:47.104901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-c261edf8-1bfa-481f-965c-6094da3c4639 None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:47.105235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-c261edf8-1bfa-481f-965c-6094da3c4639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:47.113276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188cc16-f639-4863-8154-5ca2501d6181 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:47.113597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188cc16-f639-4863-8154-5ca2501d6181 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:47.114882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188cc16-f639-4863-8154-5ca2501d6181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:47.131688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 831/1635] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:29:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:47.146782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261edf8-1bfa-481f-965c-6094da3c4639 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:47.150787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:47.159675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261edf8-1bfa-481f-965c-6094da3c4639 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:47.170267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1384a-f013-4c53-b568-99f94091e31a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:47.170940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1384a-f013-4c53-b568-99f94091e31a service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:47.171363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1384a-f013-4c53-b568-99f94091e31a service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:47.171645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1384a-f013-4c53-b568-99f94091e31a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:47.184622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8188cc16-f639-4863-8154-5ca2501d6181 None None] MFA Rules not processed for user `e91e38a712074ca5a2567da012445aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:47.185983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8188cc16-f639-4863-8154-5ca2501d6181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:47.200105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1384a-f013-4c53-b568-99f94091e31a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:47.202136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1384a-f013-4c53-b568-99f94091e31a service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:47.235103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 832/1636] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:29:47.395352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 805/1637] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:29:47.407117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641ff49-9068-457a-a655-51c279de30d1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:47.407507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641ff49-9068-457a-a655-51c279de30d1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:47.407822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641ff49-9068-457a-a655-51c279de30d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:47.445118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6641ff49-9068-457a-a655-51c279de30d1 None None] MFA Rules not processed for user `e91e38a712074ca5a2567da012445aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:47.449210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6641ff49-9068-457a-a655-51c279de30d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:47.475643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188cc16-f639-4863-8154-5ca2501d6181 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:47.476610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 833/1638] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:47.494965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641ff49-9068-457a-a655-51c279de30d1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:47.527850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641ff49-9068-457a-a655-51c279de30d1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:47.541693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d06988-6877-40aa-a46a-202cda75cfc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:47.542730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d06988-6877-40aa-a46a-202cda75cfc7 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:47.543013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d06988-6877-40aa-a46a-202cda75cfc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:47.543294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d06988-6877-40aa-a46a-202cda75cfc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:47.573332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d06988-6877-40aa-a46a-202cda75cfc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:47.576441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d06988-6877-40aa-a46a-202cda75cfc7 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:47.621490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 834/1639] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:47.651387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188cc16-f639-4863-8154-5ca2501d6181 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:47.658560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:47.659819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:47.660325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:47.660841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] PATH_INFO: `/v3/users/bd79f566bbcd4725b358eea50996447a` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:47.662323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd79f566bbcd4725b358eea50996447a)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:47.664537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:47.746081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd79f566bbcd4725b358eea50996447a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:47.746902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd79f566bbcd4725b358eea50996447a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:47.769421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd79f566bbcd4725b358eea50996447a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:47.770350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd79f566bbcd4725b358eea50996447a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:47.818411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd79f566bbcd4725b358eea50996447a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:47.843368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 806/1640] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:29:47 2022] DELETE /identity/v3/users/bd79f566bbcd4725b358eea50996447a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:29:47.854780 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d2ada-3f8c-4932-98d0-b6b7fd3c1258 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:47.855078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d2ada-3f8c-4932-98d0-b6b7fd3c1258 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:47.855342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d2ada-3f8c-4932-98d0-b6b7fd3c1258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:47.879785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1d0d2ada-3f8c-4932-98d0-b6b7fd3c1258 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:47.881224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1d0d2ada-3f8c-4932-98d0-b6b7fd3c1258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:47.980168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 835/1641] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:48.040750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:48.043559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:48.085844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:48.195161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1b7c-7585-4428-8ea8-35c1e0da51ae admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:48.199741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d6d88-b464-4cea-ac5a-d040c68b7fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:48.200678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d6d88-b464-4cea-ac5a-d040c68b7fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:48.201734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d6d88-b464-4cea-ac5a-d040c68b7fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:48.202384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d6d88-b464-4cea-ac5a-d040c68b7fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:48.237677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d6d88-b464-4cea-ac5a-d040c68b7fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:48.239570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d6d88-b464-4cea-ac5a-d040c68b7fe1 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:48.275695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 807/1642] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:48.796587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d2ada-3f8c-4932-98d0-b6b7fd3c1258 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:48.798130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1d0d2ada-3f8c-4932-98d0-b6b7fd3c1258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:48.822401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d2ada-3f8c-4932-98d0-b6b7fd3c1258 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:48.841666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:48.842577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:48.843057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:48.843507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] PATH_INFO: `/v3/users/058ee448f819446f9ff20d57b12b3b3a` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:48.845195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=058ee448f819446f9ff20d57b12b3b3a)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:48.846394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: WARNING py.warnings [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:29:48.846394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: warnings.warn(msg) May 23 15:29:48.846394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: May 23 15:29:48.848100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:48.876614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be78192-d985-4f82-9ee5-eaf3989c050a None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:48.877631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be78192-d985-4f82-9ee5-eaf3989c050a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:48.878105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be78192-d985-4f82-9ee5-eaf3989c050a None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:48.879726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 808/1643] 10.222.0.30 () {54 vars in 912 bytes} [Mon May 23 15:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:29:48.887956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b5182-d8b7-43de-bef2-408872b76b5f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:48.889326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b5182-d8b7-43de-bef2-408872b76b5f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:48.889803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b5182-d8b7-43de-bef2-408872b76b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:48.970359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ab8b5182-d8b7-43de-bef2-408872b76b5f None None] MFA Rules not processed for user `68d85e20dc884cd0bad5bb6c304776ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:48.971797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ab8b5182-d8b7-43de-bef2-408872b76b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:48.994099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 809/1644] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:29:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:29:49.000853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058ee448f819446f9ff20d57b12b3b3a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:49.041304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:49.054006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b5182-d8b7-43de-bef2-408872b76b5f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:49.063091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b5182-d8b7-43de-bef2-408872b76b5f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:49.064091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058ee448f819446f9ff20d57b12b3b3a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:49.078438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893382ae-746c-4f3e-a3b8-5768572e27a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:49.080120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893382ae-746c-4f3e-a3b8-5768572e27a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:49.080759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893382ae-746c-4f3e-a3b8-5768572e27a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:49.081418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893382ae-746c-4f3e-a3b8-5768572e27a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:49.084202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058ee448f819446f9ff20d57b12b3b3a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:49.084967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058ee448f819446f9ff20d57b12b3b3a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:49.086367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058ee448f819446f9ff20d57b12b3b3a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:49.105150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893382ae-746c-4f3e-a3b8-5768572e27a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:49.107179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893382ae-746c-4f3e-a3b8-5768572e27a6 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:49.128201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 810/1645] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:29:49.142500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 836/1646] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:29:48 2022] DELETE /identity/v3/users/058ee448f819446f9ff20d57b12b3b3a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:49.146749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cdfde-1dcc-492c-a5f1-4a4b783ab469 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:49.147273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cdfde-1dcc-492c-a5f1-4a4b783ab469 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:49.147763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cdfde-1dcc-492c-a5f1-4a4b783ab469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:49.180804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-0b4cdfde-1dcc-492c-a5f1-4a4b783ab469 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:49.182862 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0b4cdfde-1dcc-492c-a5f1-4a4b783ab469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:49.366353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 811/1647] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:49.388782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:49.391298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:49.606209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983baf4a-ad1f-4d12-89a7-cf8927a1e113 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:49.620443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea04e3-d01d-4d83-ad8d-9667514f89d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:49.623666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea04e3-d01d-4d83-ad8d-9667514f89d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:49.624382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea04e3-d01d-4d83-ad8d-9667514f89d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:49.626553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea04e3-d01d-4d83-ad8d-9667514f89d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:49.667174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea04e3-d01d-4d83-ad8d-9667514f89d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:49.691402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea04e3-d01d-4d83-ad8d-9667514f89d7 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:49.729552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 837/1648] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:50.228166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cdfde-1dcc-492c-a5f1-4a4b783ab469 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:50.229632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0b4cdfde-1dcc-492c-a5f1-4a4b783ab469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:50.255471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cdfde-1dcc-492c-a5f1-4a4b783ab469 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:50.270473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:50.272841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:50.274238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:50.275468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] PATH_INFO: `/v3/projects/da1811a65421483f9fed0285deca97a2` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:50.278294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1811a65421483f9fed0285deca97a2)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:50.282242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: WARNING py.warnings [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:29:50.282242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: warnings.warn(msg) May 23 15:29:50.282242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: May 23 15:29:50.286261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:50.439736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1811a65421483f9fed0285deca97a2', 'request_id': 'req-35c6324f-e02d-4e1d-b761-b615be141032'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:50.454006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1811a65421483f9fed0285deca97a2', 'request_id': 'req-35c6324f-e02d-4e1d-b761-b615be141032'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:50.473033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1811a65421483f9fed0285deca97a2', 'request_id': 'req-35c6324f-e02d-4e1d-b761-b615be141032'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:50.533186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] The token cache is being invalidate because project da1811a65421483f9fed0285deca97a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:29:50.538122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:50.542337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-35c6324f-e02d-4e1d-b761-b615be141032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:50.551017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 812/1649] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:29:50 2022] DELETE /identity/v3/projects/da1811a65421483f9fed0285deca97a2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:50.556916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216d42-2aa8-4604-8066-28698b901ce8 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:50.557915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216d42-2aa8-4604-8066-28698b901ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:50.558395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216d42-2aa8-4604-8066-28698b901ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:50.616440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-17216d42-2aa8-4604-8066-28698b901ce8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:50.620623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-17216d42-2aa8-4604-8066-28698b901ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:50.688694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becdccd-8f95-49d9-ac83-65fda859a529 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:50.689108 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becdccd-8f95-49d9-ac83-65fda859a529 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:50.689397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becdccd-8f95-49d9-ac83-65fda859a529 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:50.691785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 813/1650] 10.222.0.30 () {54 vars in 960 bytes} [Mon May 23 15:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:29:50.698595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d2ebb-f65a-49f1-a415-c0985895fbec None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:50.698881 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d2ebb-f65a-49f1-a415-c0985895fbec None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:50.699681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d2ebb-f65a-49f1-a415-c0985895fbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:50.777407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ae9d2ebb-f65a-49f1-a415-c0985895fbec None None] MFA Rules not processed for user `3a7dd5a0102d4d3493f09e809cb3caaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:50.778706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ae9d2ebb-f65a-49f1-a415-c0985895fbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:50.799769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:50.921119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 838/1651] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:50.926149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311ebae-cf26-467e-bd79-3eb7a791934b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:50.926149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311ebae-cf26-467e-bd79-3eb7a791934b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:50.927671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311ebae-cf26-467e-bd79-3eb7a791934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:51.024726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b311ebae-cf26-467e-bd79-3eb7a791934b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:51.026943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b311ebae-cf26-467e-bd79-3eb7a791934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:51.085530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 814/1652] 10.222.0.30 () {58 vars in 1057 bytes} [Mon May 23 15:29:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:51.093426 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 839/1653] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:51.094478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311ebae-cf26-467e-bd79-3eb7a791934b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:51.094580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d2ebb-f65a-49f1-a415-c0985895fbec None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:51.118276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d2ebb-f65a-49f1-a415-c0985895fbec None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:51.120487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311ebae-cf26-467e-bd79-3eb7a791934b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:51.133784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11016b-b790-4de5-9b73-5e2271bdbca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:51.134479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11016b-b790-4de5-9b73-5e2271bdbca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:51.134762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11016b-b790-4de5-9b73-5e2271bdbca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:51.135032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11016b-b790-4de5-9b73-5e2271bdbca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:51.136035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11016b-b790-4de5-9b73-5e2271bdbca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:51.136820 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11016b-b790-4de5-9b73-5e2271bdbca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:51.138084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11016b-b790-4de5-9b73-5e2271bdbca7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:51.149873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544af3d-de6e-4dbb-82ea-41a8ffd1197c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:51.150891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544af3d-de6e-4dbb-82ea-41a8ffd1197c service placement] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:51.151188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544af3d-de6e-4dbb-82ea-41a8ffd1197c service placement] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:51.151471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544af3d-de6e-4dbb-82ea-41a8ffd1197c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:51.161980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1544af3d-de6e-4dbb-82ea-41a8ffd1197c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:51.172435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 840/1654] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:29:51.178120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11016b-b790-4de5-9b73-5e2271bdbca7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:51.202615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544af3d-de6e-4dbb-82ea-41a8ffd1197c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:51.205396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544af3d-de6e-4dbb-82ea-41a8ffd1197c service placement] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:51.210648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11016b-b790-4de5-9b73-5e2271bdbca7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:51.225853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aca104-2dbe-414b-a694-bd0e5170cf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:51.226083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aca104-2dbe-414b-a694-bd0e5170cf83 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:51.226361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aca104-2dbe-414b-a694-bd0e5170cf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:51.227080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aca104-2dbe-414b-a694-bd0e5170cf83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:51.227618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aca104-2dbe-414b-a694-bd0e5170cf83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:51.229321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aca104-2dbe-414b-a694-bd0e5170cf83 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:51.231157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 815/1655] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:51.321615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 841/1656] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:51 2022] POST /identity/v3/projects => generated 403 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:51.326271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544af3d-de6e-4dbb-82ea-41a8ffd1197c service placement] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:51.345976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544af3d-de6e-4dbb-82ea-41a8ffd1197c service placement] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:51.484584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08753304-6043-4b15-a9df-375d57a29bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:51.485624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08753304-6043-4b15-a9df-375d57a29bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:51.485791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08753304-6043-4b15-a9df-375d57a29bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:51.486028 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08753304-6043-4b15-a9df-375d57a29bc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:51.492277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08753304-6043-4b15-a9df-375d57a29bc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:51.495371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08753304-6043-4b15-a9df-375d57a29bc7 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:51.546412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 816/1657] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:51 2022] POST /identity/v3/users => generated 302 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:51.547433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aca104-2dbe-414b-a694-bd0e5170cf83 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:51.599151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aca104-2dbe-414b-a694-bd0e5170cf83 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:51.618522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1052f6-a34a-4c93-9b2d-044b432d0471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:51.619220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1052f6-a34a-4c93-9b2d-044b432d0471 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:51.619504 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1052f6-a34a-4c93-9b2d-044b432d0471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:51.619779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1052f6-a34a-4c93-9b2d-044b432d0471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:51.620508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1052f6-a34a-4c93-9b2d-044b432d0471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:51.622146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1052f6-a34a-4c93-9b2d-044b432d0471 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:51.661720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 842/1658] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:51.664105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08753304-6043-4b15-a9df-375d57a29bc7 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:51.675698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08753304-6043-4b15-a9df-375d57a29bc7 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:51.701847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f409e-d908-438c-a1b3-96ae6bb738de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:51.702865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f409e-d908-438c-a1b3-96ae6bb738de admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:51.703309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f409e-d908-438c-a1b3-96ae6bb738de admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:51.703720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f409e-d908-438c-a1b3-96ae6bb738de admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:51.704600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f409e-d908-438c-a1b3-96ae6bb738de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:51.706729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f409e-d908-438c-a1b3-96ae6bb738de admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:51.721279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 817/1659] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:51.725130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1052f6-a34a-4c93-9b2d-044b432d0471 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:51.742851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1052f6-a34a-4c93-9b2d-044b432d0471 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:51.756939 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388de3c-34b7-4dd1-ac2a-613100803696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:51.758519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388de3c-34b7-4dd1-ac2a-613100803696 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:51.758952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388de3c-34b7-4dd1-ac2a-613100803696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:51.759395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388de3c-34b7-4dd1-ac2a-613100803696 admin admin] PATH_INFO: `/v3/projects/1c44335f677a4eaab89608f07a0c3ce9/users/1a9890023f8e44d99d32c1518c084fb0/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:52.016591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f409e-d908-438c-a1b3-96ae6bb738de admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:52.018249 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8b6f409e-d908-438c-a1b3-96ae6bb738de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:52.050058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f409e-d908-438c-a1b3-96ae6bb738de admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:52.075307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:52.077192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:52.077550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:52.077831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] PATH_INFO: `/v3/projects/65624b5cd9aa4aeead4174807d1be2e3` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:52.084563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65624b5cd9aa4aeead4174807d1be2e3)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:52.086470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:52.198144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388de3c-34b7-4dd1-ac2a-613100803696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c44335f677a4eaab89608f07a0c3ce9, user_id=1a9890023f8e44d99d32c1518c084fb0, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:52.207445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388de3c-34b7-4dd1-ac2a-613100803696 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:52.213593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65624b5cd9aa4aeead4174807d1be2e3', 'request_id': 'req-7e924cd9-a15b-498e-8217-55a523f792a3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:52.228417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65624b5cd9aa4aeead4174807d1be2e3', 'request_id': 'req-7e924cd9-a15b-498e-8217-55a523f792a3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:52.242612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 843/1660] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:51 2022] PUT /identity/v3/projects/1c44335f677a4eaab89608f07a0c3ce9/users/1a9890023f8e44d99d32c1518c084fb0/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:52.254532 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3f5b9-437f-41f0-abc8-f8a140e60e77 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:52.255021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3f5b9-437f-41f0-abc8-f8a140e60e77 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:52.255432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3f5b9-437f-41f0-abc8-f8a140e60e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:52.262949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65624b5cd9aa4aeead4174807d1be2e3', 'request_id': 'req-7e924cd9-a15b-498e-8217-55a523f792a3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:52.264796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:52.286723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:52.322681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] The token cache is being invalidate because project 65624b5cd9aa4aeead4174807d1be2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:29:52.322995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:52.327458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:52.332718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 818/1661] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:29:52 2022] DELETE /identity/v3/projects/65624b5cd9aa4aeead4174807d1be2e3 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:29:52.337001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:52.342376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:52.386517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0aa3f5b9-437f-41f0-abc8-f8a140e60e77 None None] MFA Rules not processed for user `1a9890023f8e44d99d32c1518c084fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:52.387933 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0aa3f5b9-437f-41f0-abc8-f8a140e60e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:52.539371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 844/1662] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:52.545233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3f5b9-437f-41f0-abc8-f8a140e60e77 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:52.548307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0aa3f5b9-437f-41f0-abc8-f8a140e60e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:52.583180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e924cd9-a15b-498e-8217-55a523f792a3 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:52.760365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:52.766512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:52.767666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:52.768018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] PATH_INFO: `/v3/users/c3682c6c68b24c4caaaa5b81f63dc477` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:52.772528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3682c6c68b24c4caaaa5b81f63dc477)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:52.774550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:52.878445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3f5b9-437f-41f0-abc8-f8a140e60e77 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:52.902304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244710b-a9ea-43cf-9e56-f3fc46855bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:52.906590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244710b-a9ea-43cf-9e56-f3fc46855bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:52.907786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244710b-a9ea-43cf-9e56-f3fc46855bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:52.909914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244710b-a9ea-43cf-9e56-f3fc46855bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:52.912345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244710b-a9ea-43cf-9e56-f3fc46855bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:52.917023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244710b-a9ea-43cf-9e56-f3fc46855bb3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:52.948856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3682c6c68b24c4caaaa5b81f63dc477'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:52.952220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3682c6c68b24c4caaaa5b81f63dc477'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:52.976090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3682c6c68b24c4caaaa5b81f63dc477'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:52.977749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3682c6c68b24c4caaaa5b81f63dc477'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:53.030991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3682c6c68b24c4caaaa5b81f63dc477'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:53.074253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 819/1663] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:29:52 2022] DELETE /identity/v3/users/c3682c6c68b24c4caaaa5b81f63dc477 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:53.081425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:53.083134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:53.083565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 845/1664] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:52 2022] POST /identity/v3/projects => generated 403 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:53.155034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:53.321608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7530518-412b-4538-b6e1-0ca064912f2a admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:53.518317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33adb28-1e3a-4dcd-8089-799ef79a12fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:53.519019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33adb28-1e3a-4dcd-8089-799ef79a12fa admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:53.519868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33adb28-1e3a-4dcd-8089-799ef79a12fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:53.520152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33adb28-1e3a-4dcd-8089-799ef79a12fa admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:53.520858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33adb28-1e3a-4dcd-8089-799ef79a12fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:53.525694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33adb28-1e3a-4dcd-8089-799ef79a12fa admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:53.607411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244710b-a9ea-43cf-9e56-f3fc46855bb3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:53.626616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 820/1665] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:53 2022] POST /identity/v3/users => generated 308 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:53.629144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244710b-a9ea-43cf-9e56-f3fc46855bb3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:53.663769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e53dc-e3f5-405e-989f-95a188947c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:53.665383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e53dc-e3f5-405e-989f-95a188947c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:53.665784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e53dc-e3f5-405e-989f-95a188947c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:53.675969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e53dc-e3f5-405e-989f-95a188947c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:53.677101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e53dc-e3f5-405e-989f-95a188947c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:53.679618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e53dc-e3f5-405e-989f-95a188947c3b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:53.698880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 846/1666] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:53.700943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33adb28-1e3a-4dcd-8089-799ef79a12fa admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:53.742034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33adb28-1e3a-4dcd-8089-799ef79a12fa admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:53.758922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc6e74-7ec2-4ac7-8110-ebac3fd9d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:53.760183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc6e74-7ec2-4ac7-8110-ebac3fd9d561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:53.765203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc6e74-7ec2-4ac7-8110-ebac3fd9d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:53.765784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc6e74-7ec2-4ac7-8110-ebac3fd9d561 admin admin] PATH_INFO: `/v3/projects/c332306485764c37bc7d51ebacc421a7/users/016e10a4093c4afc9a243fe5e45d5240/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:53.835956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc6e74-7ec2-4ac7-8110-ebac3fd9d561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c332306485764c37bc7d51ebacc421a7, user_id=016e10a4093c4afc9a243fe5e45d5240, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:53.839910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc6e74-7ec2-4ac7-8110-ebac3fd9d561 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:53.867476 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 821/1667] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:53 2022] PUT /identity/v3/projects/c332306485764c37bc7d51ebacc421a7/users/016e10a4093c4afc9a243fe5e45d5240/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:53.872825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e53dc-e3f5-405e-989f-95a188947c3b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:53.891926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e53dc-e3f5-405e-989f-95a188947c3b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:54.075150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8822fd1-eb5d-40de-9d28-eb86d0239cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:54.076773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822fd1-eb5d-40de-9d28-eb86d0239cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:54.077336 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822fd1-eb5d-40de-9d28-eb86d0239cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:54.077703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822fd1-eb5d-40de-9d28-eb86d0239cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:54.078595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8822fd1-eb5d-40de-9d28-eb86d0239cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:54.081577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8822fd1-eb5d-40de-9d28-eb86d0239cd0 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:54.114274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc6e74-7ec2-4ac7-8110-ebac3fd9d561 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:54.116527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 847/1668] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:54.139656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc6e74-7ec2-4ac7-8110-ebac3fd9d561 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:54.158655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4227d4-328d-45d6-971f-fed435db67d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:54.159821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4227d4-328d-45d6-971f-fed435db67d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:54.160239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4227d4-328d-45d6-971f-fed435db67d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:54.160614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4227d4-328d-45d6-971f-fed435db67d2 admin admin] PATH_INFO: `/v3/projects/c332306485764c37bc7d51ebacc421a7/users/016e10a4093c4afc9a243fe5e45d5240/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:54.164664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4227d4-328d-45d6-971f-fed435db67d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c332306485764c37bc7d51ebacc421a7, user_id=016e10a4093c4afc9a243fe5e45d5240, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:54.168855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4227d4-328d-45d6-971f-fed435db67d2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:54.200376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35286c-b11c-46e8-995c-2a7ee01f8137 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:54.204197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35286c-b11c-46e8-995c-2a7ee01f8137 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:54.204603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35286c-b11c-46e8-995c-2a7ee01f8137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:54.207259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 822/1669] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:54 2022] PUT /identity/v3/projects/c332306485764c37bc7d51ebacc421a7/users/016e10a4093c4afc9a243fe5e45d5240/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:54.231210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:54.276042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ea35286c-b11c-46e8-995c-2a7ee01f8137 None None] MFA Rules not processed for user `016e10a4093c4afc9a243fe5e45d5240`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:54.278110 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ea35286c-b11c-46e8-995c-2a7ee01f8137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:54.512828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 848/1670] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:54 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:54.524061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4227d4-328d-45d6-971f-fed435db67d2 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:54.525875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-eb4227d4-328d-45d6-971f-fed435db67d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:54.699503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4227d4-328d-45d6-971f-fed435db67d2 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:54.711318 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab30f5-b90e-4715-9162-92d4f243570a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:54.712939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab30f5-b90e-4715-9162-92d4f243570a service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:54.719711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab30f5-b90e-4715-9162-92d4f243570a service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:54.720848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab30f5-b90e-4715-9162-92d4f243570a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:54.763506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab30f5-b90e-4715-9162-92d4f243570a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:54.769354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab30f5-b90e-4715-9162-92d4f243570a service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:54.824350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 823/1671] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:54 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:55.142134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35286c-b11c-46e8-995c-2a7ee01f8137 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:55.310724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35286c-b11c-46e8-995c-2a7ee01f8137 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:55.318643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:55.319561 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:55.322652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:55.323660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] PATH_INFO: `/v3/users/1a9890023f8e44d99d32c1518c084fb0` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:55.326934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a9890023f8e44d99d32c1518c084fb0)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:55.334221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:55.437287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a9890023f8e44d99d32c1518c084fb0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:55.463605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a9890023f8e44d99d32c1518c084fb0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:55.476042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a9890023f8e44d99d32c1518c084fb0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:55.477849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a9890023f8e44d99d32c1518c084fb0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:55.480240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a9890023f8e44d99d32c1518c084fb0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:55.503617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 849/1672] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:29:55 2022] DELETE /identity/v3/users/1a9890023f8e44d99d32c1518c084fb0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:55.508697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc85d-9041-4208-a51d-bff65cd751f7 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:55.509006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc85d-9041-4208-a51d-bff65cd751f7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:55.509280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc85d-9041-4208-a51d-bff65cd751f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:55.535226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-719dc85d-9041-4208-a51d-bff65cd751f7 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:55.537410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-719dc85d-9041-4208-a51d-bff65cd751f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:55.579935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:55.731723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 824/1673] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:55.798702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:55.800276 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:55.827560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:55.917904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca87abc-b64e-4f83-8486-b970e6601749 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:55.925393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee449e-dd0f-4a95-ae34-abb10908aa78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:55.926291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee449e-dd0f-4a95-ae34-abb10908aa78 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:55.926756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee449e-dd0f-4a95-ae34-abb10908aa78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:55.927422 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee449e-dd0f-4a95-ae34-abb10908aa78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:55.944602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee449e-dd0f-4a95-ae34-abb10908aa78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:55.946469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee449e-dd0f-4a95-ae34-abb10908aa78 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:55.965628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 850/1674] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:29:56.201604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf1522-068a-43c5-86eb-56b9f6d51688 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:56.201604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf1522-068a-43c5-86eb-56b9f6d51688 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:56.201604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf1522-068a-43c5-86eb-56b9f6d51688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:56.254418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-74cf1522-068a-43c5-86eb-56b9f6d51688 None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:56.255848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-74cf1522-068a-43c5-86eb-56b9f6d51688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:56.266697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee449e-dd0f-4a95-ae34-abb10908aa78 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:56.268355 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-51ee449e-dd0f-4a95-ae34-abb10908aa78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:56.288107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee449e-dd0f-4a95-ae34-abb10908aa78 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:56.317558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:56.318457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:56.318744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:56.319022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] PATH_INFO: `/v3/projects/9ebc5b8a5fc64b6aa20fd12a5cc8899b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:56.330189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ebc5b8a5fc64b6aa20fd12a5cc8899b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:56.332192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:56.489255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 825/1675] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:29:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:56.505010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ebc5b8a5fc64b6aa20fd12a5cc8899b', 'request_id': 'req-d1183726-8e6f-4fb7-ae7c-f69390fc369f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:56.538692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ebc5b8a5fc64b6aa20fd12a5cc8899b', 'request_id': 'req-d1183726-8e6f-4fb7-ae7c-f69390fc369f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:56.554616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ebc5b8a5fc64b6aa20fd12a5cc8899b', 'request_id': 'req-d1183726-8e6f-4fb7-ae7c-f69390fc369f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:56.587801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] The token cache is being invalidate because project 9ebc5b8a5fc64b6aa20fd12a5cc8899b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:29:56.588381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:56.589275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:56.600275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46cbf-c476-47de-880b-9fa2753098e8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:56.600762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46cbf-c476-47de-880b-9fa2753098e8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:56.600992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46cbf-c476-47de-880b-9fa2753098e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:56.603067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 851/1676] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:29:56 2022] DELETE /identity/v3/projects/9ebc5b8a5fc64b6aa20fd12a5cc8899b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:56.637751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-61f46cbf-c476-47de-880b-9fa2753098e8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:56.643164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-61f46cbf-c476-47de-880b-9fa2753098e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:56.821439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:56.823385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:56.982283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 826/1677] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:56.983982 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af0f3b-f9a0-4f26-a31a-5494cd2b6612 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:56.985187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af0f3b-f9a0-4f26-a31a-5494cd2b6612 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:56.985576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af0f3b-f9a0-4f26-a31a-5494cd2b6612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:57.093083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-42af0f3b-f9a0-4f26-a31a-5494cd2b6612 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:57.095891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-42af0f3b-f9a0-4f26-a31a-5494cd2b6612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:57.142329 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 827/1678] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:57.153898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af0f3b-f9a0-4f26-a31a-5494cd2b6612 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:57.203770 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af0f3b-f9a0-4f26-a31a-5494cd2b6612 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:57.222910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1183726-8e6f-4fb7-ae7c-f69390fc369f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:57.226792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42dfa6-828b-45b4-b34b-8030a9f302fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:57.227676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42dfa6-828b-45b4-b34b-8030a9f302fe admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:57.228081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42dfa6-828b-45b4-b34b-8030a9f302fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:57.228457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42dfa6-828b-45b4-b34b-8030a9f302fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:57.229796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42dfa6-828b-45b4-b34b-8030a9f302fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:29:57.230539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42dfa6-828b-45b4-b34b-8030a9f302fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:57.233872 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42dfa6-828b-45b4-b34b-8030a9f302fe admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:57.248364 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b91496-fea3-48c5-8a90-45cded9fc6b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:57.249212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b91496-fea3-48c5-8a90-45cded9fc6b9 service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:57.249604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b91496-fea3-48c5-8a90-45cded9fc6b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:57.249988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b91496-fea3-48c5-8a90-45cded9fc6b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:57.257744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-75b91496-fea3-48c5-8a90-45cded9fc6b9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:57.299985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 828/1679] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:29:57.302100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42dfa6-828b-45b4-b34b-8030a9f302fe admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:57.339853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42dfa6-828b-45b4-b34b-8030a9f302fe admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:57.366494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904462c5-c475-4318-8e61-9a0c11732da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:57.367357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904462c5-c475-4318-8e61-9a0c11732da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:57.367746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904462c5-c475-4318-8e61-9a0c11732da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:57.368130 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904462c5-c475-4318-8e61-9a0c11732da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:57.368993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904462c5-c475-4318-8e61-9a0c11732da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:57.371145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904462c5-c475-4318-8e61-9a0c11732da8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:57.415415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:57.496701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 829/1680] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:57 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:57.501969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904462c5-c475-4318-8e61-9a0c11732da8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:57.512257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b91496-fea3-48c5-8a90-45cded9fc6b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:57.519396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b91496-fea3-48c5-8a90-45cded9fc6b9 service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:57.541670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 852/1681] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:29:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 721 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 15:29:57.559304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:57.685973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904462c5-c475-4318-8e61-9a0c11732da8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:57.703404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf4203-d286-4484-a598-b71f0280f804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:57.704435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf4203-d286-4484-a598-b71f0280f804 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:57.704887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf4203-d286-4484-a598-b71f0280f804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:57.705303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf4203-d286-4484-a598-b71f0280f804 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:57.706149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf4203-d286-4484-a598-b71f0280f804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:57.708655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf4203-d286-4484-a598-b71f0280f804 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:57.751446 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 830/1682] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:57 2022] POST /identity/v3/users => generated 291 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:57.756554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b91496-fea3-48c5-8a90-45cded9fc6b9 service swift] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:57.774259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b91496-fea3-48c5-8a90-45cded9fc6b9 service swift] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:57.810362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91bc03-04b8-4d66-b0f0-3e559268a7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:57.811201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91bc03-04b8-4d66-b0f0-3e559268a7bb admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:57.811551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91bc03-04b8-4d66-b0f0-3e559268a7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:57.812064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91bc03-04b8-4d66-b0f0-3e559268a7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:57.816425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91bc03-04b8-4d66-b0f0-3e559268a7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:57.836723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91bc03-04b8-4d66-b0f0-3e559268a7bb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:57.862187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 853/1683] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:57.867368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf4203-d286-4484-a598-b71f0280f804 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:57.880574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf4203-d286-4484-a598-b71f0280f804 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:57.898986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f5fb9-62dd-4387-ae73-4a452420b245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:57.899917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f5fb9-62dd-4387-ae73-4a452420b245 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:57.900648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f5fb9-62dd-4387-ae73-4a452420b245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:57.900815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f5fb9-62dd-4387-ae73-4a452420b245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:57.902315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f5fb9-62dd-4387-ae73-4a452420b245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:57.904380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f5fb9-62dd-4387-ae73-4a452420b245 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:57.935127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91bc03-04b8-4d66-b0f0-3e559268a7bb admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:57.956988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91bc03-04b8-4d66-b0f0-3e559268a7bb admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:57.957363 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 831/1684] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:57.974857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3741de-63f1-43b8-9217-939cf4954a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:57.975960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3741de-63f1-43b8-9217-939cf4954a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:57.976368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3741de-63f1-43b8-9217-939cf4954a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:57.976757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3741de-63f1-43b8-9217-939cf4954a0f admin admin] PATH_INFO: `/v3/projects/36c1edd9b2234789b61bad42bd29304e/users/5b2b5bc9f538444ba5ac928233d0feab/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:58.030129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3741de-63f1-43b8-9217-939cf4954a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c1edd9b2234789b61bad42bd29304e, user_id=5b2b5bc9f538444ba5ac928233d0feab, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:58.034310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3741de-63f1-43b8-9217-939cf4954a0f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:58.091588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 854/1685] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:57 2022] PUT /identity/v3/projects/36c1edd9b2234789b61bad42bd29304e/users/5b2b5bc9f538444ba5ac928233d0feab/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:58.093654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9382c-a06a-4a60-b1f8-a81066802c5c None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:58.094097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9382c-a06a-4a60-b1f8-a81066802c5c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:58.094512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9382c-a06a-4a60-b1f8-a81066802c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:58.176419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-15f9382c-a06a-4a60-b1f8-a81066802c5c None None] MFA Rules not processed for user `5b2b5bc9f538444ba5ac928233d0feab`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:58.178285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-15f9382c-a06a-4a60-b1f8-a81066802c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:58.342472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 832/1686] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:58.347441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3741de-63f1-43b8-9217-939cf4954a0f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:58.566953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3741de-63f1-43b8-9217-939cf4954a0f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:58.578367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6943cb-8a21-41ab-bb28-88a2ef6e6210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:58.579385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6943cb-8a21-41ab-bb28-88a2ef6e6210 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:58.579844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6943cb-8a21-41ab-bb28-88a2ef6e6210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:58.580259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6943cb-8a21-41ab-bb28-88a2ef6e6210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:58.581137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6943cb-8a21-41ab-bb28-88a2ef6e6210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:58.586217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6943cb-8a21-41ab-bb28-88a2ef6e6210 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:58.658083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 855/1687] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:29:58 2022] POST /identity/v3/projects => generated 381 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:29:58.662083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9382c-a06a-4a60-b1f8-a81066802c5c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:58.677521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9382c-a06a-4a60-b1f8-a81066802c5c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:58.801590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1f0ca-fffa-40a0-888c-96fc722f8b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:58.803026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1f0ca-fffa-40a0-888c-96fc722f8b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:58.803448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1f0ca-fffa-40a0-888c-96fc722f8b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:58.803860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1f0ca-fffa-40a0-888c-96fc722f8b05 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:58.804694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1f0ca-fffa-40a0-888c-96fc722f8b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:58.806963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1f0ca-fffa-40a0-888c-96fc722f8b05 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:58.865987 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 833/1688] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:29:58 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:29:58.870417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6943cb-8a21-41ab-bb28-88a2ef6e6210 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:58.890747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6943cb-8a21-41ab-bb28-88a2ef6e6210 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:58.898328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dede1-eaab-4aa6-a73a-06b4f50d23a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:58.900913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dede1-eaab-4aa6-a73a-06b4f50d23a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:58.903193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dede1-eaab-4aa6-a73a-06b4f50d23a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:58.903912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dede1-eaab-4aa6-a73a-06b4f50d23a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:58.905118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383dede1-eaab-4aa6-a73a-06b4f50d23a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:58.908484 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383dede1-eaab-4aa6-a73a-06b4f50d23a2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:58.939594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 856/1689] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:58.944200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1f0ca-fffa-40a0-888c-96fc722f8b05 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:58.961651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1f0ca-fffa-40a0-888c-96fc722f8b05 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:58.980609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fbe78-09a1-4988-ab75-691b7cc55385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:58.982114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fbe78-09a1-4988-ab75-691b7cc55385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:58.982836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fbe78-09a1-4988-ab75-691b7cc55385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:58.983415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fbe78-09a1-4988-ab75-691b7cc55385 admin admin] PATH_INFO: `/v3/projects/35ea8a7f8528418990c0db13e74d4b0f/users/c653f3f762a24a6b98c4bae0bbfe1f40/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:59.026694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fbe78-09a1-4988-ab75-691b7cc55385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ea8a7f8528418990c0db13e74d4b0f, user_id=c653f3f762a24a6b98c4bae0bbfe1f40, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:59.030846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fbe78-09a1-4988-ab75-691b7cc55385 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:59.050482 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:59.079068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 834/1690] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:58 2022] PUT /identity/v3/projects/35ea8a7f8528418990c0db13e74d4b0f/users/c653f3f762a24a6b98c4bae0bbfe1f40/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:59.083060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dede1-eaab-4aa6-a73a-06b4f50d23a2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:59.102845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dede1-eaab-4aa6-a73a-06b4f50d23a2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:59.198206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fbe78-09a1-4988-ab75-691b7cc55385 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:59.205863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-147fbe78-09a1-4988-ab75-691b7cc55385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:59.239606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e818962-2c37-4334-a490-ad449960a35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:59.242029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e818962-2c37-4334-a490-ad449960a35d admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:59.242709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e818962-2c37-4334-a490-ad449960a35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:59.243157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e818962-2c37-4334-a490-ad449960a35d admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:59.246660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e818962-2c37-4334-a490-ad449960a35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:59.250017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e818962-2c37-4334-a490-ad449960a35d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:59.275018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 857/1691] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:29:59.278094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e818962-2c37-4334-a490-ad449960a35d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:59.304897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e818962-2c37-4334-a490-ad449960a35d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:59.318170 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab47802-309e-49ec-9ff5-a277d89e23ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:59.320168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab47802-309e-49ec-9ff5-a277d89e23ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:59.322351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab47802-309e-49ec-9ff5-a277d89e23ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:59.322351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab47802-309e-49ec-9ff5-a277d89e23ae admin admin] PATH_INFO: `/v3/projects/35ea8a7f8528418990c0db13e74d4b0f/users/c653f3f762a24a6b98c4bae0bbfe1f40/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:59.328361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab47802-309e-49ec-9ff5-a277d89e23ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ea8a7f8528418990c0db13e74d4b0f, user_id=c653f3f762a24a6b98c4bae0bbfe1f40, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:59.334868 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab47802-309e-49ec-9ff5-a277d89e23ae admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:59.356811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:29:59.400973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fbe78-09a1-4988-ab75-691b7cc55385 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:59.413055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 858/1692] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:29:59 2022] PUT /identity/v3/projects/35ea8a7f8528418990c0db13e74d4b0f/users/c653f3f762a24a6b98c4bae0bbfe1f40/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:29:59.435989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243a2f3-cd02-4b13-82a4-b923bb7a9315 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:59.436348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243a2f3-cd02-4b13-82a4-b923bb7a9315 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:59.436699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243a2f3-cd02-4b13-82a4-b923bb7a9315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:59.558608 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-2243a2f3-cd02-4b13-82a4-b923bb7a9315 None None] MFA Rules not processed for user `c653f3f762a24a6b98c4bae0bbfe1f40`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:29:59.565063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2243a2f3-cd02-4b13-82a4-b923bb7a9315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:59.634950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:29:59.636606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:29:59.637123 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:29:59.637717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] PATH_INFO: `/v3/users/8776d23acd7048faabbafa37a3028dd6` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:29:59.639726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8776d23acd7048faabbafa37a3028dd6)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:29:59.642076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:29:59.742332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8776d23acd7048faabbafa37a3028dd6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:59.743974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8776d23acd7048faabbafa37a3028dd6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:59.762155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8776d23acd7048faabbafa37a3028dd6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:59.763911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8776d23acd7048faabbafa37a3028dd6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:59.804766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8776d23acd7048faabbafa37a3028dd6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:29:59.828278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 859/1693] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:29:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:29:59.853012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243a2f3-cd02-4b13-82a4-b923bb7a9315 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:59.856038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 835/1694] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:29:59 2022] DELETE /identity/v3/users/8776d23acd7048faabbafa37a3028dd6 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:29:59.856038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:29:59.865824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2243a2f3-cd02-4b13-82a4-b923bb7a9315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:59.866856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:29:59.903015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243a2f3-cd02-4b13-82a4-b923bb7a9315 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:29:59.964208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:00.052255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cbc03-4413-4c1f-a659-510020168159 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:00.065923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2c9d6-13a5-438a-b893-2121615e4edd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:00.067689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2c9d6-13a5-438a-b893-2121615e4edd service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:00.067980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2c9d6-13a5-438a-b893-2121615e4edd service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:00.068255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2c9d6-13a5-438a-b893-2121615e4edd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:00.099307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:00.100354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:00.100730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:00.101137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] PATH_INFO: `/v3/projects/1c44335f677a4eaab89608f07a0c3ce9` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:00.111275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c44335f677a4eaab89608f07a0c3ce9)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:00.113533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:00.242815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2c9d6-13a5-438a-b893-2121615e4edd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:00.245823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2c9d6-13a5-438a-b893-2121615e4edd service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:00.246400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c44335f677a4eaab89608f07a0c3ce9', 'request_id': 'req-04334a75-4102-402d-a480-1dfa5f5672a7'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.265962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c44335f677a4eaab89608f07a0c3ce9', 'request_id': 'req-04334a75-4102-402d-a480-1dfa5f5672a7'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.273059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 836/1695] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:29:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:00.276105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3854d4f-ff79-4a18-8165-55d10022312c None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:00.277357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3854d4f-ff79-4a18-8165-55d10022312c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:00.277646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3854d4f-ff79-4a18-8165-55d10022312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:00.303921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c44335f677a4eaab89608f07a0c3ce9', 'request_id': 'req-04334a75-4102-402d-a480-1dfa5f5672a7'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.331525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b3854d4f-ff79-4a18-8165-55d10022312c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:00.335088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b3854d4f-ff79-4a18-8165-55d10022312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:00.351952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] The token cache is being invalidate because project 1c44335f677a4eaab89608f07a0c3ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:00.353198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.356959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.374783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 860/1696] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:29:59 2022] DELETE /identity/v3/projects/1c44335f677a4eaab89608f07a0c3ce9 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:00.376951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:00.383170 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:00.420492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:00.496543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 837/1697] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:00.532924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3854d4f-ff79-4a18-8165-55d10022312c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:00.534697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b3854d4f-ff79-4a18-8165-55d10022312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:00.583537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04334a75-4102-402d-a480-1dfa5f5672a7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:00.604914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:00.605826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:00.606101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:00.606422 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] PATH_INFO: `/v3/users/016e10a4093c4afc9a243fe5e45d5240` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:00.610837 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] RBAC: Authorizing `identity:delete_user(user_id=016e10a4093c4afc9a243fe5e45d5240)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:00.612825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:00.690154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3854d4f-ff79-4a18-8165-55d10022312c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:00.703927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc474803-9336-4aca-86a0-98f81938e042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:00.704730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc474803-9336-4aca-86a0-98f81938e042 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:00.704961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc474803-9336-4aca-86a0-98f81938e042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:00.705360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc474803-9336-4aca-86a0-98f81938e042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:00.730403 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc474803-9336-4aca-86a0-98f81938e042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:00.734837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc474803-9336-4aca-86a0-98f81938e042 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:00.777512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 838/1698] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:00.824566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016e10a4093c4afc9a243fe5e45d5240'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.856180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425eab5-696d-43b9-a72d-afa0bf2c97f5 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:00.857826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425eab5-696d-43b9-a72d-afa0bf2c97f5 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:00.858122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425eab5-696d-43b9-a72d-afa0bf2c97f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:00.911188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016e10a4093c4afc9a243fe5e45d5240'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.933287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016e10a4093c4afc9a243fe5e45d5240'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.933962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016e10a4093c4afc9a243fe5e45d5240'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.935268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016e10a4093c4afc9a243fe5e45d5240'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:00.937597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d425eab5-696d-43b9-a72d-afa0bf2c97f5 None None] MFA Rules not processed for user `68d85e20dc884cd0bad5bb6c304776ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:00.943748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d425eab5-696d-43b9-a72d-afa0bf2c97f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:00.988731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 839/1699] 10.222.0.72 () {58 vars in 1015 bytes} [Mon May 23 15:30:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:00.995922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 861/1700] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:00 2022] DELETE /identity/v3/users/016e10a4093c4afc9a243fe5e45d5240 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:01.006592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425eab5-696d-43b9-a72d-afa0bf2c97f5 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:01.008033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d425eab5-696d-43b9-a72d-afa0bf2c97f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:01.037062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:01.075003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:01.077060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:01.147795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425eab5-696d-43b9-a72d-afa0bf2c97f5 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:01.291737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945b663-983b-4cf6-bb2f-2046319742df admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:01.299707 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be8380-7872-42b6-b1aa-2090cf8c5c32 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c653f3f762a24a6b98c4bae0bbfe1f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35ea8a7f8528418990c0db13e74d4b0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:01.300022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead6e3e-0332-4c85-a9e7-5b9affa378f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:01.300710 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead6e3e-0332-4c85-a9e7-5b9affa378f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:01.301042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead6e3e-0332-4c85-a9e7-5b9affa378f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:01.301326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead6e3e-0332-4c85-a9e7-5b9affa378f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:01.302097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be8380-7872-42b6-b1aa-2090cf8c5c32 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:01.303910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be8380-7872-42b6-b1aa-2090cf8c5c32 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:01.304383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be8380-7872-42b6-b1aa-2090cf8c5c32 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:01.305354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be8380-7872-42b6-b1aa-2090cf8c5c32 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:01.311639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be8380-7872-42b6-b1aa-2090cf8c5c32 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:01.314131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be8380-7872-42b6-b1aa-2090cf8c5c32 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:01.339596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead6e3e-0332-4c85-a9e7-5b9affa378f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:01.342098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead6e3e-0332-4c85-a9e7-5b9affa378f0 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:01.353573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 840/1701] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:01.356612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be8380-7872-42b6-b1aa-2090cf8c5c32 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:01.391688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 862/1702] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:01.393311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be8380-7872-42b6-b1aa-2090cf8c5c32 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:01.416668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3941e-93ab-4d91-ad6d-3988b4ac6c60 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c653f3f762a24a6b98c4bae0bbfe1f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35ea8a7f8528418990c0db13e74d4b0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:01.417364 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3941e-93ab-4d91-ad6d-3988b4ac6c60 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:01.417655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3941e-93ab-4d91-ad6d-3988b4ac6c60 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:01.417940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3941e-93ab-4d91-ad6d-3988b4ac6c60 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:01.423015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3941e-93ab-4d91-ad6d-3988b4ac6c60 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:01.429923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3941e-93ab-4d91-ad6d-3988b4ac6c60 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:01.537011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 841/1703] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:01 2022] POST /identity/v3/projects => generated 354 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:01.538919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead6e3e-0332-4c85-a9e7-5b9affa378f0 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:01.572188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead6e3e-0332-4c85-a9e7-5b9affa378f0 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:01.673493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:01.757195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c653f3f762a24a6b98c4bae0bbfe1f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35ea8a7f8528418990c0db13e74d4b0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:01.758274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:01.759348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:01.760052 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:01.761249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:01.761776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:01.764443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:01.779333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 863/1704] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:01.871849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3941e-93ab-4d91-ad6d-3988b4ac6c60 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:01.891035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3941e-93ab-4d91-ad6d-3988b4ac6c60 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:01.902808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c653f3f762a24a6b98c4bae0bbfe1f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35ea8a7f8528418990c0db13e74d4b0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:01.906622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:01.906914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:01.907194 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] PATH_INFO: `/v3/projects/f8fbe8e31dec4e4e82dc9bb592b3f77c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:01.911526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f8fbe8e31dec4e4e82dc9bb592b3f77c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:01.923214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:02.100264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8fbe8e31dec4e4e82dc9bb592b3f77c', 'request_id': 'req-410e0477-840c-4447-b2de-18aa4ceaf6f4'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:02.121436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8fbe8e31dec4e4e82dc9bb592b3f77c', 'request_id': 'req-410e0477-840c-4447-b2de-18aa4ceaf6f4'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:02.152299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8fbe8e31dec4e4e82dc9bb592b3f77c', 'request_id': 'req-410e0477-840c-4447-b2de-18aa4ceaf6f4'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:02.188385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] The token cache is being invalidate because project f8fbe8e31dec4e4e82dc9bb592b3f77c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:02.188684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:02.193724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:02.199705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 842/1705] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:01 2022] DELETE /identity/v3/projects/f8fbe8e31dec4e4e82dc9bb592b3f77c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:02.812955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:02.817068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:02.841317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:02.843199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:02.898513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:03.042021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807f489-98c3-426b-ad72-3a3488051ef2 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:03.104595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e0477-840c-4447-b2de-18aa4ceaf6f4 tempest-VolumeQuotaClassesTest-1082800510 tempest-VolumeQuotaClassesTest-1082800510-project-admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:03.120968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:03.123377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:03.124577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:03.124577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] PATH_INFO: `/v3/users/5b2b5bc9f538444ba5ac928233d0feab` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:03.131624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2b5bc9f538444ba5ac928233d0feab)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:03.137608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:03.190431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:03.194501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:03.194793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:03.195130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] PATH_INFO: `/v3/projects/c332306485764c37bc7d51ebacc421a7` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:03.198625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c332306485764c37bc7d51ebacc421a7)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:03.202267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:03.261947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2b5bc9f538444ba5ac928233d0feab'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.262843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2b5bc9f538444ba5ac928233d0feab'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.282019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2b5bc9f538444ba5ac928233d0feab'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.285252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2b5bc9f538444ba5ac928233d0feab'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.318550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3a1d540d-33e5-4ecb-bb88-5fca54d3557c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2b5bc9f538444ba5ac928233d0feab'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.353772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c332306485764c37bc7d51ebacc421a7', 'request_id': 'req-1a25958c-0880-46ef-b089-86b5e7667e18'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.358550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 843/1706] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:02 2022] DELETE /identity/v3/users/5b2b5bc9f538444ba5ac928233d0feab => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:03.360732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce604d-06f0-48c4-b5c2-44a01aacf487 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:03.362359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce604d-06f0-48c4-b5c2-44a01aacf487 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:03.362834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce604d-06f0-48c4-b5c2-44a01aacf487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:03.365993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c332306485764c37bc7d51ebacc421a7', 'request_id': 'req-1a25958c-0880-46ef-b089-86b5e7667e18'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.376350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c332306485764c37bc7d51ebacc421a7', 'request_id': 'req-1a25958c-0880-46ef-b089-86b5e7667e18'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.417646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] The token cache is being invalidate because project c332306485764c37bc7d51ebacc421a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:03.418611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.420101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1a25958c-0880-46ef-b089-86b5e7667e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:03.422836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 864/1707] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:02 2022] DELETE /identity/v3/projects/c332306485764c37bc7d51ebacc421a7 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:03.433022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1bbd4-7800-44d1-8d3c-b5b96d9d0990 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:03.433022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1bbd4-7800-44d1-8d3c-b5b96d9d0990 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:03.433370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1bbd4-7800-44d1-8d3c-b5b96d9d0990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:03.437114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-31ce604d-06f0-48c4-b5c2-44a01aacf487 None None] MFA Rules not processed for user `139e3e42acff4c4ca8df0a7c4fc81053`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:03.438762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-31ce604d-06f0-48c4-b5c2-44a01aacf487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:03.481002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:03.522370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-99c1bbd4-7800-44d1-8d3c-b5b96d9d0990 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:03.529615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-99c1bbd4-7800-44d1-8d3c-b5b96d9d0990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:03.640293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 844/1708] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:03.645097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39e98-0ac9-4d7d-864f-92c46a0ea86e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:03.645429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39e98-0ac9-4d7d-864f-92c46a0ea86e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:03.645657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39e98-0ac9-4d7d-864f-92c46a0ea86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:03.722917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-62c39e98-0ac9-4d7d-864f-92c46a0ea86e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:03.732849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-62c39e98-0ac9-4d7d-864f-92c46a0ea86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:03.776291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 865/1709] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:03.778109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1bbd4-7800-44d1-8d3c-b5b96d9d0990 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:03.783709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-99c1bbd4-7800-44d1-8d3c-b5b96d9d0990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:03.800945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:03.945241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 845/1710] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:03.945241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39e98-0ac9-4d7d-864f-92c46a0ea86e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:03.956412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-62c39e98-0ac9-4d7d-864f-92c46a0ea86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:04.081287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1bbd4-7800-44d1-8d3c-b5b96d9d0990 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:04.125983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee26fe3-1a0b-40ac-81f5-941b286de206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:04.128745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26fe3-1a0b-40ac-81f5-941b286de206 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:04.129254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26fe3-1a0b-40ac-81f5-941b286de206 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:04.129698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26fe3-1a0b-40ac-81f5-941b286de206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:04.179349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee26fe3-1a0b-40ac-81f5-941b286de206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:04.187402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee26fe3-1a0b-40ac-81f5-941b286de206 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:04.242609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 866/1711] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:04.269649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574574d5-756b-4cd4-91dc-2128a0456c33 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:04.269937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574574d5-756b-4cd4-91dc-2128a0456c33 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:04.270218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574574d5-756b-4cd4-91dc-2128a0456c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:04.300413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39e98-0ac9-4d7d-864f-92c46a0ea86e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:04.329986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f190b-8c9f-412f-9454-1a7f20828e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:04.330788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f190b-8c9f-412f-9454-1a7f20828e0b service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:04.331082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f190b-8c9f-412f-9454-1a7f20828e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:04.331365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f190b-8c9f-412f-9454-1a7f20828e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:04.396640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-574574d5-756b-4cd4-91dc-2128a0456c33 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:04.397985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-574574d5-756b-4cd4-91dc-2128a0456c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:04.405388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f190b-8c9f-412f-9454-1a7f20828e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:04.406895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f190b-8c9f-412f-9454-1a7f20828e0b service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:04.455834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 867/1712] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:04.466973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574574d5-756b-4cd4-91dc-2128a0456c33 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:04.484685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 846/1713] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:04.597225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574574d5-756b-4cd4-91dc-2128a0456c33 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:04.680448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01a6aa-9d9b-4fc1-853d-1c362a1d4ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:04.681267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01a6aa-9d9b-4fc1-853d-1c362a1d4ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:04.681562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01a6aa-9d9b-4fc1-853d-1c362a1d4ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:04.681867 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01a6aa-9d9b-4fc1-853d-1c362a1d4ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:04.682683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01a6aa-9d9b-4fc1-853d-1c362a1d4ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:04.682987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01a6aa-9d9b-4fc1-853d-1c362a1d4ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:04.696285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01a6aa-9d9b-4fc1-853d-1c362a1d4ca7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:04.753666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f190b-8c9f-412f-9454-1a7f20828e0b service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:04.756679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0a3f190b-8c9f-412f-9454-1a7f20828e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:04.777827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 868/1714] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:04.779449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01a6aa-9d9b-4fc1-853d-1c362a1d4ca7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:04.814179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f190b-8c9f-412f-9454-1a7f20828e0b service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:04.840119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:04.841714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:04.842036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:04.842357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] PATH_INFO: `/v3/projects/1aebe2f4fe924c0093c13e4f92070f0b` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:04.842467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01a6aa-9d9b-4fc1-853d-1c362a1d4ca7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:04.867632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aebe2f4fe924c0093c13e4f92070f0b)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:04.867846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bfa07-4111-43e9-9a20-ce23f2c6f043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:04.868365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bfa07-4111-43e9-9a20-ce23f2c6f043 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:04.868653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bfa07-4111-43e9-9a20-ce23f2c6f043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:04.868963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bfa07-4111-43e9-9a20-ce23f2c6f043 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:04.869625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:04.869732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06bfa07-4111-43e9-9a20-ce23f2c6f043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:04.892901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06bfa07-4111-43e9-9a20-ce23f2c6f043 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:05.004149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 869/1715] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:04 2022] POST /identity/v3/projects => generated 381 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:05.009054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bfa07-4111-43e9-9a20-ce23f2c6f043 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:05.060860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aebe2f4fe924c0093c13e4f92070f0b', 'request_id': 'req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:05.085509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aebe2f4fe924c0093c13e4f92070f0b', 'request_id': 'req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:05.109218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aebe2f4fe924c0093c13e4f92070f0b', 'request_id': 'req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:05.155004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] The token cache is being invalidate because project 1aebe2f4fe924c0093c13e4f92070f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:05.155299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:05.158685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ad217e52-aac2-4b3a-86e4-2abd5666ea7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:05.163225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 847/1716] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:04 2022] DELETE /identity/v3/projects/1aebe2f4fe924c0093c13e4f92070f0b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:05.173400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c68db-27c8-4d7a-ac99-e477e07fe7ec None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:05.173693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c68db-27c8-4d7a-ac99-e477e07fe7ec None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:05.173983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c68db-27c8-4d7a-ac99-e477e07fe7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:05.213396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:05.250849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-080c68db-27c8-4d7a-ac99-e477e07fe7ec None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:05.253415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-080c68db-27c8-4d7a-ac99-e477e07fe7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:05.256064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bfa07-4111-43e9-9a20-ce23f2c6f043 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:05.463226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:05.545581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 848/1717] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:05.554243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb263701-6d41-4478-b27f-2211f0848926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:05.555178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb263701-6d41-4478-b27f-2211f0848926 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:05.556847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43954a6d-d97a-44c0-aa32-4c7a2e6c6b3c None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:05.557358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb263701-6d41-4478-b27f-2211f0848926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:05.557774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb263701-6d41-4478-b27f-2211f0848926 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:05.558654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb263701-6d41-4478-b27f-2211f0848926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:05.559101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43954a6d-d97a-44c0-aa32-4c7a2e6c6b3c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:05.560140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43954a6d-d97a-44c0-aa32-4c7a2e6c6b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:05.565640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb263701-6d41-4478-b27f-2211f0848926 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:05.712423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 870/1718] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:05 2022] POST /identity/v3/users => generated 291 bytes in 700 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:05.714023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb263701-6d41-4478-b27f-2211f0848926 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:05.718550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-43954a6d-d97a-44c0-aa32-4c7a2e6c6b3c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:05.719585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-cb263701-6d41-4478-b27f-2211f0848926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:05.721026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-43954a6d-d97a-44c0-aa32-4c7a2e6c6b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:05.754666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 849/1719] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:05.755830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43954a6d-d97a-44c0-aa32-4c7a2e6c6b3c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:05.766170 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb263701-6d41-4478-b27f-2211f0848926 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:05.785730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad884e-ab5b-4611-9cbe-275aeda871ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:05.790020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad884e-ab5b-4611-9cbe-275aeda871ba admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:05.791890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad884e-ab5b-4611-9cbe-275aeda871ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:05.792655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad884e-ab5b-4611-9cbe-275aeda871ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:05.796988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43954a6d-d97a-44c0-aa32-4c7a2e6c6b3c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:05.799140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad884e-ab5b-4611-9cbe-275aeda871ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:05.805284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad884e-ab5b-4611-9cbe-275aeda871ba admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:05.819990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8041316-5d47-44e1-8cc1-85a5d459fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:05.820919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8041316-5d47-44e1-8cc1-85a5d459fc4c admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:05.821319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8041316-5d47-44e1-8cc1-85a5d459fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:05.822037 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8041316-5d47-44e1-8cc1-85a5d459fc4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:05.824112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8041316-5d47-44e1-8cc1-85a5d459fc4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:05.825684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8041316-5d47-44e1-8cc1-85a5d459fc4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:05.828517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8041316-5d47-44e1-8cc1-85a5d459fc4c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:05.851777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 871/1720] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:05.853037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad884e-ab5b-4611-9cbe-275aeda871ba admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:05.885933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 850/1721] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:05.890481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8041316-5d47-44e1-8cc1-85a5d459fc4c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:05.895077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad884e-ab5b-4611-9cbe-275aeda871ba admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:05.925089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3d668-d6dd-4b14-b21e-49c152a5d9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:05.926254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3d668-d6dd-4b14-b21e-49c152a5d9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:05.926677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3d668-d6dd-4b14-b21e-49c152a5d9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:05.927106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3d668-d6dd-4b14-b21e-49c152a5d9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:05.930039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3d668-d6dd-4b14-b21e-49c152a5d9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:05.931822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3d668-d6dd-4b14-b21e-49c152a5d9c8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:05.935336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8041316-5d47-44e1-8cc1-85a5d459fc4c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:05.957672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38de4f-fa19-4bad-9e47-7aab898c4605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:05.961243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38de4f-fa19-4bad-9e47-7aab898c4605 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:05.961767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38de4f-fa19-4bad-9e47-7aab898c4605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:05.962485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38de4f-fa19-4bad-9e47-7aab898c4605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:05.966773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38de4f-fa19-4bad-9e47-7aab898c4605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:05.983635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38de4f-fa19-4bad-9e47-7aab898c4605 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:05.987351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 872/1722] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:05.988839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3d668-d6dd-4b14-b21e-49c152a5d9c8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:06.027875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3d668-d6dd-4b14-b21e-49c152a5d9c8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:06.070004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544da177-d0d9-4e07-aa30-6a24e4abf899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:06.073989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544da177-d0d9-4e07-aa30-6a24e4abf899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:06.079629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544da177-d0d9-4e07-aa30-6a24e4abf899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:06.080076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544da177-d0d9-4e07-aa30-6a24e4abf899 admin admin] PATH_INFO: `/v3/projects/3ca4934988684a0e809719aab3af2feb/users/27c9fa59b70b44a9b73a547a8bf44848/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:06.143775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 851/1723] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:05 2022] POST /identity/v3/projects => generated 401 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:06.153146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38de4f-fa19-4bad-9e47-7aab898c4605 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:06.173217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544da177-d0d9-4e07-aa30-6a24e4abf899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca4934988684a0e809719aab3af2feb, user_id=27c9fa59b70b44a9b73a547a8bf44848, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:06.195683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544da177-d0d9-4e07-aa30-6a24e4abf899 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:06.242847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 873/1724] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:05 2022] PUT /identity/v3/projects/3ca4934988684a0e809719aab3af2feb/users/27c9fa59b70b44a9b73a547a8bf44848/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:06.258089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97934d00-0b14-403e-ae42-4bc35e0e6d26 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:06.258576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97934d00-0b14-403e-ae42-4bc35e0e6d26 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:06.259048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97934d00-0b14-403e-ae42-4bc35e0e6d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:06.373476 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-97934d00-0b14-403e-ae42-4bc35e0e6d26 None None] MFA Rules not processed for user `27c9fa59b70b44a9b73a547a8bf44848`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:06.375426 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-97934d00-0b14-403e-ae42-4bc35e0e6d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:06.401861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38de4f-fa19-4bad-9e47-7aab898c4605 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:06.553849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c6ee5-c0f1-4aea-9b11-cbfffa820f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:06.558370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c6ee5-c0f1-4aea-9b11-cbfffa820f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:06.558370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c6ee5-c0f1-4aea-9b11-cbfffa820f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:06.558370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c6ee5-c0f1-4aea-9b11-cbfffa820f40 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:06.558370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c6ee5-c0f1-4aea-9b11-cbfffa820f40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:06.563070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c6ee5-c0f1-4aea-9b11-cbfffa820f40 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:06.644573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 852/1725] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:06 2022] POST /identity/v3/users => generated 301 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:06.645428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c6ee5-c0f1-4aea-9b11-cbfffa820f40 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:06.657296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c6ee5-c0f1-4aea-9b11-cbfffa820f40 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:06.665718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f206df2-3f1d-45d0-9c93-91682b354be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:06.668222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f206df2-3f1d-45d0-9c93-91682b354be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:06.671602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f206df2-3f1d-45d0-9c93-91682b354be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:06.672172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f206df2-3f1d-45d0-9c93-91682b354be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:06.673826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f206df2-3f1d-45d0-9c93-91682b354be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:06.690527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f206df2-3f1d-45d0-9c93-91682b354be2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:06.702776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 874/1726] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:06.704743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b199-3138-4eaa-be11-ea6e977bb16b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:06.705472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b199-3138-4eaa-be11-ea6e977bb16b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:06.705762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b199-3138-4eaa-be11-ea6e977bb16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:06.736395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 853/1727] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:06.744126 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f206df2-3f1d-45d0-9c93-91682b354be2 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:06.769179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f206df2-3f1d-45d0-9c93-91682b354be2 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:06.784811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9b48f-e7c1-48af-80f5-904c6aaaa782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:06.785805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9b48f-e7c1-48af-80f5-904c6aaaa782 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:06.786301 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9b48f-e7c1-48af-80f5-904c6aaaa782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:06.788995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9b48f-e7c1-48af-80f5-904c6aaaa782 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:06.789998 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9b48f-e7c1-48af-80f5-904c6aaaa782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:06.796967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9b48f-e7c1-48af-80f5-904c6aaaa782 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:06.823448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1190b199-3138-4eaa-be11-ea6e977bb16b None None] MFA Rules not processed for user `5074b9fa045543b9a62bf751e736d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:06.827873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1190b199-3138-4eaa-be11-ea6e977bb16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:06.936927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:06.948480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 854/1728] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:06 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:06.967203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9b48f-e7c1-48af-80f5-904c6aaaa782 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:07.011874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:07.096151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 875/1729] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:07.113347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85415-8639-4b81-8cfb-aed6f3d867f0 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:07.113640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85415-8639-4b81-8cfb-aed6f3d867f0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:07.113921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85415-8639-4b81-8cfb-aed6f3d867f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:07.153079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9b48f-e7c1-48af-80f5-904c6aaaa782 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:07.189864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067caca-9702-45c1-a0fb-48d707c7c5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:07.190353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067caca-9702-45c1-a0fb-48d707c7c5ca admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:07.190647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067caca-9702-45c1-a0fb-48d707c7c5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:07.190924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067caca-9702-45c1-a0fb-48d707c7c5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:07.198955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067caca-9702-45c1-a0fb-48d707c7c5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:07.211611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067caca-9702-45c1-a0fb-48d707c7c5ca admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:07.242396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-48f85415-8639-4b81-8cfb-aed6f3d867f0 None None] MFA Rules not processed for user `06431315dc8040cbbafbe83a3ed8e0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:07.250584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-48f85415-8639-4b81-8cfb-aed6f3d867f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:07.257242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 855/1730] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:07.258905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067caca-9702-45c1-a0fb-48d707c7c5ca admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:07.287985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067caca-9702-45c1-a0fb-48d707c7c5ca admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:07.317854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f34d2-13a6-41d6-b105-a4b1916b3891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:07.319927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f34d2-13a6-41d6-b105-a4b1916b3891 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:07.320217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f34d2-13a6-41d6-b105-a4b1916b3891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:07.320502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f34d2-13a6-41d6-b105-a4b1916b3891 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:07.321320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f34d2-13a6-41d6-b105-a4b1916b3891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:07.326444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f34d2-13a6-41d6-b105-a4b1916b3891 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:07.402915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 856/1731] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:07 2022] POST /identity/v3/users => generated 297 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:07.418255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4640872-9f3a-46a2-a06a-1b6c20cd0d6e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:07.418544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4640872-9f3a-46a2-a06a-1b6c20cd0d6e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:07.419392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4640872-9f3a-46a2-a06a-1b6c20cd0d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:07.480979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a4640872-9f3a-46a2-a06a-1b6c20cd0d6e None None] MFA Rules not processed for user `5074b9fa045543b9a62bf751e736d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:07.482776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a4640872-9f3a-46a2-a06a-1b6c20cd0d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:07.500784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 876/1732] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:07.501753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f85415-8639-4b81-8cfb-aed6f3d867f0 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:07.561834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f85415-8639-4b81-8cfb-aed6f3d867f0 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:07.581822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35b9df-7c49-41bd-919b-b222653135fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:07.582986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35b9df-7c49-41bd-919b-b222653135fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:07.583477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35b9df-7c49-41bd-919b-b222653135fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:07.583759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35b9df-7c49-41bd-919b-b222653135fe admin admin] PATH_INFO: `/v3/projects/f721837584ae40a99b0415e997f5a44f/users/f95b81c7059c4159a65b040f3015344b/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:07.644355 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35b9df-7c49-41bd-919b-b222653135fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f721837584ae40a99b0415e997f5a44f, user_id=f95b81c7059c4159a65b040f3015344b, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:07.647838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35b9df-7c49-41bd-919b-b222653135fe admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:07.685589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 877/1733] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:07 2022] PUT /identity/v3/projects/f721837584ae40a99b0415e997f5a44f/users/f95b81c7059c4159a65b040f3015344b/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:07.689226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35b9df-7c49-41bd-919b-b222653135fe admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:07.691282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 857/1734] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:07.701866 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4640872-9f3a-46a2-a06a-1b6c20cd0d6e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:07.709072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a4640872-9f3a-46a2-a06a-1b6c20cd0d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:07.901496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35b9df-7c49-41bd-919b-b222653135fe admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:07.922639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b46a8-9f27-49d0-bae4-cf7d05680773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:07.923340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b46a8-9f27-49d0-bae4-cf7d05680773 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:07.926450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b46a8-9f27-49d0-bae4-cf7d05680773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:07.926784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b46a8-9f27-49d0-bae4-cf7d05680773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:07.938290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b46a8-9f27-49d0-bae4-cf7d05680773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:07.944619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b46a8-9f27-49d0-bae4-cf7d05680773 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:07.963856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4640872-9f3a-46a2-a06a-1b6c20cd0d6e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:07.978823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 878/1735] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:07.980406 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f9987-867b-4165-9e96-53294e7df3d5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:07.980406 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f9987-867b-4165-9e96-53294e7df3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:07.980705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f9987-867b-4165-9e96-53294e7df3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:08.010640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:08.075214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99d040-a06a-4f04-aca1-960d6c51b314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:08.076054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99d040-a06a-4f04-aca1-960d6c51b314 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:08.076344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99d040-a06a-4f04-aca1-960d6c51b314 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:08.076677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99d040-a06a-4f04-aca1-960d6c51b314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:08.088284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-fd6f9987-867b-4165-9e96-53294e7df3d5 None None] MFA Rules not processed for user `f95b81c7059c4159a65b040f3015344b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:08.097568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fd6f9987-867b-4165-9e96-53294e7df3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:08.119721 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:08.273248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99d040-a06a-4f04-aca1-960d6c51b314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:08.274961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99d040-a06a-4f04-aca1-960d6c51b314 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:08.317005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 858/1736] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:08.319427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99d040-a06a-4f04-aca1-960d6c51b314 service nova] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:08.322908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8f99d040-a06a-4f04-aca1-960d6c51b314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:08.326269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 879/1737] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:08.327207 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f9987-867b-4165-9e96-53294e7df3d5 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:08.361781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f9987-867b-4165-9e96-53294e7df3d5 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:08.376539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9dfb9-3498-4ad9-a50f-a4c86a2f212f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:08.377649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9dfb9-3498-4ad9-a50f-a4c86a2f212f admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:08.377935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9dfb9-3498-4ad9-a50f-a4c86a2f212f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:08.378211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9dfb9-3498-4ad9-a50f-a4c86a2f212f admin admin] PATH_INFO: `/v3/projects/ae8e79e46ed4453e8ee3a16498afe044/users/3384561c68dd44a3bc2bb6377306fce8/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:08.392646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99d040-a06a-4f04-aca1-960d6c51b314 service nova] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:08.418329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9dfb9-3498-4ad9-a50f-a4c86a2f212f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae8e79e46ed4453e8ee3a16498afe044, user_id=3384561c68dd44a3bc2bb6377306fce8, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:08.419096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86152fb6-167f-4e59-8124-a9b714267d53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:08.419656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152fb6-167f-4e59-8124-a9b714267d53 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:08.420128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152fb6-167f-4e59-8124-a9b714267d53 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:08.420411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152fb6-167f-4e59-8124-a9b714267d53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:08.428987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9dfb9-3498-4ad9-a50f-a4c86a2f212f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:08.460257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 880/1738] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:08 2022] PUT /identity/v3/projects/ae8e79e46ed4453e8ee3a16498afe044/users/3384561c68dd44a3bc2bb6377306fce8/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:08.461830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c18d26-6b70-4d65-9f27-4c52d0c2adf1 None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:08.461830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c18d26-6b70-4d65-9f27-4c52d0c2adf1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:08.461830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c18d26-6b70-4d65-9f27-4c52d0c2adf1 None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:08.466707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 881/1739] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:30:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:30:08.469898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c18d26-6b70-4d65-9f27-4c52d0c2adf1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:08.650692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c18d26-6b70-4d65-9f27-4c52d0c2adf1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:08.664349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffda93-a9a4-4937-9644-941e55927c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:08.665481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffda93-a9a4-4937-9644-941e55927c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:08.665989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffda93-a9a4-4937-9644-941e55927c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:08.666519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffda93-a9a4-4937-9644-941e55927c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:08.668437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffda93-a9a4-4937-9644-941e55927c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:08.680722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffda93-a9a4-4937-9644-941e55927c2f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:08.742069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 882/1740] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:08 2022] POST /identity/v3/projects => generated 401 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:08.743437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffda93-a9a4-4937-9644-941e55927c2f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:08.772900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86152fb6-167f-4e59-8124-a9b714267d53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:08.774575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86152fb6-167f-4e59-8124-a9b714267d53 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:08.819119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 859/1741] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:08.843716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f326e5-3bbd-4843-8359-45ad313bf0c8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:08.843908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f326e5-3bbd-4843-8359-45ad313bf0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:08.844192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f326e5-3bbd-4843-8359-45ad313bf0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:08.926751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-34f326e5-3bbd-4843-8359-45ad313bf0c8 None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:08.928001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-34f326e5-3bbd-4843-8359-45ad313bf0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:08.940045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffda93-a9a4-4937-9644-941e55927c2f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:08.959907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e01d8-8ff5-4b96-b586-8f89839e1a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:08.961246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e01d8-8ff5-4b96-b586-8f89839e1a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:08.961778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e01d8-8ff5-4b96-b586-8f89839e1a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:08.962143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e01d8-8ff5-4b96-b586-8f89839e1a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:08.963111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e01d8-8ff5-4b96-b586-8f89839e1a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:08.964898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e01d8-8ff5-4b96-b586-8f89839e1a2b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:08.989429 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 883/1742] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:08.990637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e01d8-8ff5-4b96-b586-8f89839e1a2b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:09.016031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e01d8-8ff5-4b96-b586-8f89839e1a2b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:09.032287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcac194-6866-4889-8e5a-5da048c0b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:09.033059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcac194-6866-4889-8e5a-5da048c0b18a admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:09.033377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcac194-6866-4889-8e5a-5da048c0b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:09.033626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcac194-6866-4889-8e5a-5da048c0b18a admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:09.034395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcac194-6866-4889-8e5a-5da048c0b18a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:09.036630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcac194-6866-4889-8e5a-5da048c0b18a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:09.061551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:09.089813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 884/1743] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:08 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:09.089813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcac194-6866-4889-8e5a-5da048c0b18a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:09.113686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcac194-6866-4889-8e5a-5da048c0b18a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:09.129592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419943a-d5cc-4b59-bcac-7be5c65e1bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:09.130660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419943a-d5cc-4b59-bcac-7be5c65e1bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:09.131037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419943a-d5cc-4b59-bcac-7be5c65e1bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:09.131352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419943a-d5cc-4b59-bcac-7be5c65e1bc1 admin admin] PATH_INFO: `/v3/projects/ae8e79e46ed4453e8ee3a16498afe044/users/3384561c68dd44a3bc2bb6377306fce8/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:09.137818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419943a-d5cc-4b59-bcac-7be5c65e1bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae8e79e46ed4453e8ee3a16498afe044, user_id=3384561c68dd44a3bc2bb6377306fce8, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:09.141494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419943a-d5cc-4b59-bcac-7be5c65e1bc1 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:09.172027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 885/1744] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:09 2022] PUT /identity/v3/projects/ae8e79e46ed4453e8ee3a16498afe044/users/3384561c68dd44a3bc2bb6377306fce8/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:09.173021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419943a-d5cc-4b59-bcac-7be5c65e1bc1 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:09.280959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 860/1745] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:09.282496 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df4060-bf91-401a-92a9-6b4b42b4fa7a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:09.308723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df4060-bf91-401a-92a9-6b4b42b4fa7a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:09.308723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df4060-bf91-401a-92a9-6b4b42b4fa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:09.317791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:09.426741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419943a-d5cc-4b59-bcac-7be5c65e1bc1 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:09.447042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8aaa8b-607b-4f55-98ff-adaf37d1e045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:09.447916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8aaa8b-607b-4f55-98ff-adaf37d1e045 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:09.448091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8aaa8b-607b-4f55-98ff-adaf37d1e045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:09.448370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8aaa8b-607b-4f55-98ff-adaf37d1e045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:09.449297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8aaa8b-607b-4f55-98ff-adaf37d1e045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:09.451143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8aaa8b-607b-4f55-98ff-adaf37d1e045 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:09.453392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-f6df4060-bf91-401a-92a9-6b4b42b4fa7a None None] MFA Rules not processed for user `3384561c68dd44a3bc2bb6377306fce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:09.456207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f6df4060-bf91-401a-92a9-6b4b42b4fa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:09.500744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 886/1746] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:09.508489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8aaa8b-607b-4f55-98ff-adaf37d1e045 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:09.546088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8aaa8b-607b-4f55-98ff-adaf37d1e045 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:09.560632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd397b6-6d24-42f0-ac29-46acd6a4c658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:09.561825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd397b6-6d24-42f0-ac29-46acd6a4c658 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:09.562601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd397b6-6d24-42f0-ac29-46acd6a4c658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:09.562889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd397b6-6d24-42f0-ac29-46acd6a4c658 admin admin] PATH_INFO: `/v3/projects/f6967ddf216b4f409cd4fccd3b51e212/users/1c2abbfb0fdd4922a17cd47d0765f29a/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:09.614347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd397b6-6d24-42f0-ac29-46acd6a4c658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6967ddf216b4f409cd4fccd3b51e212, user_id=1c2abbfb0fdd4922a17cd47d0765f29a, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:09.618003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd397b6-6d24-42f0-ac29-46acd6a4c658 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:09.656222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 887/1747] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:09 2022] PUT /identity/v3/projects/f6967ddf216b4f409cd4fccd3b51e212/users/1c2abbfb0fdd4922a17cd47d0765f29a/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:09.658230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd397b6-6d24-42f0-ac29-46acd6a4c658 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:09.866027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 861/1748] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 584 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:09.873414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df4060-bf91-401a-92a9-6b4b42b4fa7a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:09.889699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd397b6-6d24-42f0-ac29-46acd6a4c658 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:09.897031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae177820-f215-4e1f-8002-12221abdff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:09.899791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae177820-f215-4e1f-8002-12221abdff51 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:09.902184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae177820-f215-4e1f-8002-12221abdff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:09.907004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae177820-f215-4e1f-8002-12221abdff51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:09.908112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae177820-f215-4e1f-8002-12221abdff51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:09.911341 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae177820-f215-4e1f-8002-12221abdff51 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:09.954317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 888/1749] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:09.956745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae177820-f215-4e1f-8002-12221abdff51 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:09.962643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ae177820-f215-4e1f-8002-12221abdff51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:10.138650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:10.164972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae177820-f215-4e1f-8002-12221abdff51 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:10.181509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c692cb-9345-481c-9d52-998c5423ac7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:10.183653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c692cb-9345-481c-9d52-998c5423ac7c service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:10.184020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c692cb-9345-481c-9d52-998c5423ac7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:10.185257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c692cb-9345-481c-9d52-998c5423ac7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:10.194150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df4060-bf91-401a-92a9-6b4b42b4fa7a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:10.380155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c692cb-9345-481c-9d52-998c5423ac7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:10.383432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c692cb-9345-481c-9d52-998c5423ac7c service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:10.431071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 889/1750] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:10.437148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c692cb-9345-481c-9d52-998c5423ac7c service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:10.494230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442749bb-9f93-47fe-9106-a6f26d7e2721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:10.494892 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442749bb-9f93-47fe-9106-a6f26d7e2721 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:10.495132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442749bb-9f93-47fe-9106-a6f26d7e2721 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:10.495415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442749bb-9f93-47fe-9106-a6f26d7e2721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:10.519069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c692cb-9345-481c-9d52-998c5423ac7c service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:10.544283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442749bb-9f93-47fe-9106-a6f26d7e2721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:10.547642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442749bb-9f93-47fe-9106-a6f26d7e2721 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:10.559482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f3b57-6ab2-4c2d-9331-3b0e610cf59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:10.571589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f3b57-6ab2-4c2d-9331-3b0e610cf59b admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:10.571903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f3b57-6ab2-4c2d-9331-3b0e610cf59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:10.592497 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f3b57-6ab2-4c2d-9331-3b0e610cf59b admin admin] PATH_INFO: `/v3/projects/f6967ddf216b4f409cd4fccd3b51e212/users/1c2abbfb0fdd4922a17cd47d0765f29a/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:10.593321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 862/1751] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:10.603531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f3b57-6ab2-4c2d-9331-3b0e610cf59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6967ddf216b4f409cd4fccd3b51e212, user_id=1c2abbfb0fdd4922a17cd47d0765f29a, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:10.613344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442749bb-9f93-47fe-9106-a6f26d7e2721 service nova] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:10.615302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-442749bb-9f93-47fe-9106-a6f26d7e2721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:10.632281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f3b57-6ab2-4c2d-9331-3b0e610cf59b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:10.665912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442749bb-9f93-47fe-9106-a6f26d7e2721 service nova] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:10.680909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:10.689985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:10.690766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:10.694039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] PATH_INFO: `/v3/projects/36c1edd9b2234789b61bad42bd29304e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:10.703496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 890/1752] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:10 2022] PUT /identity/v3/projects/f6967ddf216b4f409cd4fccd3b51e212/users/1c2abbfb0fdd4922a17cd47d0765f29a/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:10.713328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36c1edd9b2234789b61bad42bd29304e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:10.714091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8b90b-8fb7-4fc6-9524-c469a08d8553 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:10.714388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8b90b-8fb7-4fc6-9524-c469a08d8553 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:10.715636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:10.718988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8b90b-8fb7-4fc6-9524-c469a08d8553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:10.751124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:10.882016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ebf8b90b-8fb7-4fc6-9524-c469a08d8553 None None] MFA Rules not processed for user `1c2abbfb0fdd4922a17cd47d0765f29a`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:10.883298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ebf8b90b-8fb7-4fc6-9524-c469a08d8553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:10.943939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36c1edd9b2234789b61bad42bd29304e', 'request_id': 'req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:10.967805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36c1edd9b2234789b61bad42bd29304e', 'request_id': 'req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:10.990380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36c1edd9b2234789b61bad42bd29304e', 'request_id': 'req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:11.035864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] The token cache is being invalidate because project 36c1edd9b2234789b61bad42bd29304e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:11.036174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:11.041035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:11.046339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 863/1753] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:10 2022] DELETE /identity/v3/projects/36c1edd9b2234789b61bad42bd29304e => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:11.047508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:11.055104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:11.219079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb129da-b0f7-497f-b92e-cd6c3cffd1c3 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:11.231139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:11.232071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:11.232514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:11.233361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:11.238540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 891/1754] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:11.239275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:11.240070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8b90b-8fb7-4fc6-9524-c469a08d8553 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:11.244414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ebf8b90b-8fb7-4fc6-9524-c469a08d8553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:11.482326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:11.488481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:11.541312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 864/1755] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:11.541997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:11.553413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:11.564795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8b90b-8fb7-4fc6-9524-c469a08d8553 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:11.584904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:11.586680 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:11.586980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:11.587259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] PATH_INFO: `/v3/users/c653f3f762a24a6b98c4bae0bbfe1f40` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:11.594010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c653f3f762a24a6b98c4bae0bbfe1f40)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:11.600573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:11.602125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff52879-f9b2-4b21-b684-28da439902ac service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:11.614781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059ba37-60ac-4abf-96fa-1c79ca751520 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:11.615832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059ba37-60ac-4abf-96fa-1c79ca751520 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:11.616124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059ba37-60ac-4abf-96fa-1c79ca751520 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:11.619625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059ba37-60ac-4abf-96fa-1c79ca751520 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:11.635628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:11.661560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059ba37-60ac-4abf-96fa-1c79ca751520 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:11.666704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059ba37-60ac-4abf-96fa-1c79ca751520 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:11.698944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 865/1756] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:11 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:11.754965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c653f3f762a24a6b98c4bae0bbfe1f40'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:11.788328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c653f3f762a24a6b98c4bae0bbfe1f40'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:11.809495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c653f3f762a24a6b98c4bae0bbfe1f40'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:11.810230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c653f3f762a24a6b98c4bae0bbfe1f40'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:11.810841 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-88da48b3-6bfa-4992-a30f-25f941f73f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c653f3f762a24a6b98c4bae0bbfe1f40'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:11.842707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 892/1757] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:11 2022] DELETE /identity/v3/users/c653f3f762a24a6b98c4bae0bbfe1f40 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:11.908537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059ba37-60ac-4abf-96fa-1c79ca751520 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:11.924923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059ba37-60ac-4abf-96fa-1c79ca751520 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:12.113172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78645268-0302-4fcb-891e-d460b6983b4e None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:12.113487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78645268-0302-4fcb-891e-d460b6983b4e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:12.113774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78645268-0302-4fcb-891e-d460b6983b4e None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:12.115793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 893/1758] 10.222.0.30 () {54 vars in 912 bytes} [Mon May 23 15:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:30:12.142298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff906f8-c7c0-4c52-8817-4c049985f201 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:12.142298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff906f8-c7c0-4c52-8817-4c049985f201 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:12.142298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff906f8-c7c0-4c52-8817-4c049985f201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:12.178368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:12.192784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-5ff906f8-c7c0-4c52-8817-4c049985f201 None None] MFA Rules not processed for user `68d85e20dc884cd0bad5bb6c304776ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:12.195112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5ff906f8-c7c0-4c52-8817-4c049985f201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:12.243694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b46b0f-a8c5-43bb-867a-0ec6445d7a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:12.244574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b46b0f-a8c5-43bb-867a-0ec6445d7a30 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:12.244675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b46b0f-a8c5-43bb-867a-0ec6445d7a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:12.244980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b46b0f-a8c5-43bb-867a-0ec6445d7a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:12.255916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d8b46b0f-a8c5-43bb-867a-0ec6445d7a30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:12.432550 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 894/1759] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 23 15:30:12.479069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b46b0f-a8c5-43bb-867a-0ec6445d7a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:12.483420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b46b0f-a8c5-43bb-867a-0ec6445d7a30 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:12.505692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 866/1760] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:12.513388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff906f8-c7c0-4c52-8817-4c049985f201 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:12.515018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5ff906f8-c7c0-4c52-8817-4c049985f201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:12.531204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff906f8-c7c0-4c52-8817-4c049985f201 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:12.549965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a775aa-42e9-41f1-a364-f62e237b8c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:12.550685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a775aa-42e9-41f1-a364-f62e237b8c48 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:12.550936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a775aa-42e9-41f1-a364-f62e237b8c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:12.551216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a775aa-42e9-41f1-a364-f62e237b8c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:12.581314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a775aa-42e9-41f1-a364-f62e237b8c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:12.594847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a775aa-42e9-41f1-a364-f62e237b8c48 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:12.643592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 895/1761] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:13.507564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6e96-fbfe-4731-a591-ede8a5fc9d8a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:13.508089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6e96-fbfe-4731-a591-ede8a5fc9d8a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:13.508172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6e96-fbfe-4731-a591-ede8a5fc9d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:13.595713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b50a6e96-fbfe-4731-a591-ede8a5fc9d8a None None] MFA Rules not processed for user `d0ae34a029c14c6f826456414ac0670d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:13.597528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b50a6e96-fbfe-4731-a591-ede8a5fc9d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:13.785165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a775aa-42e9-41f1-a364-f62e237b8c48 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:13.786648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f2a775aa-42e9-41f1-a364-f62e237b8c48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:13.789857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 867/1762] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:13.808455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65ae84-b35e-4c0c-9f8b-f01a7a01ebf4 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:13.809049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65ae84-b35e-4c0c-9f8b-f01a7a01ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:13.810225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65ae84-b35e-4c0c-9f8b-f01a7a01ebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:13.899421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ad65ae84-b35e-4c0c-9f8b-f01a7a01ebf4 None None] MFA Rules not processed for user `139e3e42acff4c4ca8df0a7c4fc81053`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:13.905021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ad65ae84-b35e-4c0c-9f8b-f01a7a01ebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:14.075373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 868/1763] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:14.076141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a775aa-42e9-41f1-a364-f62e237b8c48 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:14.079797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65ae84-b35e-4c0c-9f8b-f01a7a01ebf4 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:14.091191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ad65ae84-b35e-4c0c-9f8b-f01a7a01ebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:14.101639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614f1c0-4a0b-4df6-8a69-8aa180bd32f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:14.102415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614f1c0-4a0b-4df6-8a69-8aa180bd32f9 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:14.102594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614f1c0-4a0b-4df6-8a69-8aa180bd32f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:14.102873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614f1c0-4a0b-4df6-8a69-8aa180bd32f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:14.135467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614f1c0-4a0b-4df6-8a69-8aa180bd32f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:14.137469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614f1c0-4a0b-4df6-8a69-8aa180bd32f9 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:14.167650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 896/1764] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:14.188494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:14.306040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65ae84-b35e-4c0c-9f8b-f01a7a01ebf4 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:14.351927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:14.352835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:14.355773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:14.356119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] PATH_INFO: `/v3/projects/35ea8a7f8528418990c0db13e74d4b0f` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:14.360491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ea8a7f8528418990c0db13e74d4b0f)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:14.378144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:14.519096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ea8a7f8528418990c0db13e74d4b0f', 'request_id': 'req-fd0c0077-afe4-44ed-aef4-133e98744a3f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:14.529080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ea8a7f8528418990c0db13e74d4b0f', 'request_id': 'req-fd0c0077-afe4-44ed-aef4-133e98744a3f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:14.545647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ea8a7f8528418990c0db13e74d4b0f', 'request_id': 'req-fd0c0077-afe4-44ed-aef4-133e98744a3f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:14.562582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] The token cache is being invalidate because project 35ea8a7f8528418990c0db13e74d4b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:14.563163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:14.564505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fd0c0077-afe4-44ed-aef4-133e98744a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:14.567535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 869/1765] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:14 2022] DELETE /identity/v3/projects/35ea8a7f8528418990c0db13e74d4b0f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:14.572959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81918d5-d1fd-448f-adf2-3e6f53f82a08 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:14.573330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81918d5-d1fd-448f-adf2-3e6f53f82a08 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:14.577014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81918d5-d1fd-448f-adf2-3e6f53f82a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:14.598321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:14.606078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a81918d5-d1fd-448f-adf2-3e6f53f82a08 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:14.608153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a81918d5-d1fd-448f-adf2-3e6f53f82a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:14.783652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 897/1766] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:14.789415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4338e-2883-4582-9c26-ffb27bb39ef5 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:14.789713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4338e-2883-4582-9c26-ffb27bb39ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:14.789996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4338e-2883-4582-9c26-ffb27bb39ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:14.851609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-2fc4338e-2883-4582-9c26-ffb27bb39ef5 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:14.851609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-2fc4338e-2883-4582-9c26-ffb27bb39ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:14.876153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 870/1767] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:14.878148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81918d5-d1fd-448f-adf2-3e6f53f82a08 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:14.895587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81918d5-d1fd-448f-adf2-3e6f53f82a08 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:14.917692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a7fb3-1a96-4a8d-9f43-0b4f2330ee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:14.919273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a7fb3-1a96-4a8d-9f43-0b4f2330ee95 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:14.931811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a7fb3-1a96-4a8d-9f43-0b4f2330ee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:14.932451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a7fb3-1a96-4a8d-9f43-0b4f2330ee95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:14.934826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a7fb3-1a96-4a8d-9f43-0b4f2330ee95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:14.935379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a7fb3-1a96-4a8d-9f43-0b4f2330ee95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:14.939445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a7fb3-1a96-4a8d-9f43-0b4f2330ee95 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:15.006470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 898/1768] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:15.010325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4338e-2883-4582-9c26-ffb27bb39ef5 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:15.031870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4338e-2883-4582-9c26-ffb27bb39ef5 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:15.064803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46d6fe-191f-42be-96fc-5f619c0298f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:15.065523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46d6fe-191f-42be-96fc-5f619c0298f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:15.065801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46d6fe-191f-42be-96fc-5f619c0298f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:15.066078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46d6fe-191f-42be-96fc-5f619c0298f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:15.075102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46d6fe-191f-42be-96fc-5f619c0298f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:15.095393 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46d6fe-191f-42be-96fc-5f619c0298f0 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:15.217266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 871/1769] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:15 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:15.221544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a7fb3-1a96-4a8d-9f43-0b4f2330ee95 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:15.275411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a7fb3-1a96-4a8d-9f43-0b4f2330ee95 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:15.771499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912f057-6533-4fed-a64d-93295aec9cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:15.778694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912f057-6533-4fed-a64d-93295aec9cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:15.779203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912f057-6533-4fed-a64d-93295aec9cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:15.780884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912f057-6533-4fed-a64d-93295aec9cc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:15.784091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912f057-6533-4fed-a64d-93295aec9cc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:15.810024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912f057-6533-4fed-a64d-93295aec9cc2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:15.891864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46d6fe-191f-42be-96fc-5f619c0298f0 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:15.892819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 899/1770] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:15 2022] POST /identity/v3/users => generated 293 bytes in 672 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:15.921537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46d6fe-191f-42be-96fc-5f619c0298f0 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:15.940023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d98fb-bfe6-4a95-bd57-8ab71c22ba40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:15.942144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d98fb-bfe6-4a95-bd57-8ab71c22ba40 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:15.944447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d98fb-bfe6-4a95-bd57-8ab71c22ba40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:15.945132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d98fb-bfe6-4a95-bd57-8ab71c22ba40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:15.946036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d98fb-bfe6-4a95-bd57-8ab71c22ba40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:15.952612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d98fb-bfe6-4a95-bd57-8ab71c22ba40 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:16.055929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 872/1771] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:16.060723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912f057-6533-4fed-a64d-93295aec9cc2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:16.086338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912f057-6533-4fed-a64d-93295aec9cc2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:16.120749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b683b93-bb76-4240-841a-ef3dbe79f857 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:16.121295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b683b93-bb76-4240-841a-ef3dbe79f857 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:16.122375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db31a39-96b3-4836-a92b-7b1b111804ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:16.122846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b683b93-bb76-4240-841a-ef3dbe79f857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:16.123704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db31a39-96b3-4836-a92b-7b1b111804ed admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:16.125236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db31a39-96b3-4836-a92b-7b1b111804ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:16.127235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db31a39-96b3-4836-a92b-7b1b111804ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:16.129715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db31a39-96b3-4836-a92b-7b1b111804ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:16.136352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db31a39-96b3-4836-a92b-7b1b111804ed admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:16.167489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 900/1772] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:16.169917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db31a39-96b3-4836-a92b-7b1b111804ed admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:16.171633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-0b683b93-bb76-4240-841a-ef3dbe79f857 None None] MFA Rules not processed for user `5074b9fa045543b9a62bf751e736d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:16.172981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0b683b93-bb76-4240-841a-ef3dbe79f857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:16.197345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db31a39-96b3-4836-a92b-7b1b111804ed admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:16.237934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5c6d2-bd69-4a70-998b-51171f3582f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:16.243659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5c6d2-bd69-4a70-998b-51171f3582f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:16.247883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5c6d2-bd69-4a70-998b-51171f3582f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:16.251264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5c6d2-bd69-4a70-998b-51171f3582f7 admin admin] PATH_INFO: `/v3/projects/42ec461cb12e4710a19d58ae6fc4fbc3/users/87e4323175b54c8e9adad6929827c1f3/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:16.330272 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5c6d2-bd69-4a70-998b-51171f3582f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ec461cb12e4710a19d58ae6fc4fbc3, user_id=87e4323175b54c8e9adad6929827c1f3, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:16.347394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5c6d2-bd69-4a70-998b-51171f3582f7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:16.375746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 873/1773] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:16.441345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 901/1774] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:16 2022] PUT /identity/v3/projects/42ec461cb12e4710a19d58ae6fc4fbc3/users/87e4323175b54c8e9adad6929827c1f3/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:16.448389 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5fc67-ef92-4f0e-89dc-266614c22bfd None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:16.448609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5fc67-ef92-4f0e-89dc-266614c22bfd None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:16.448953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5fc67-ef92-4f0e-89dc-266614c22bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:16.534351 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-f3b5fc67-ef92-4f0e-89dc-266614c22bfd None None] MFA Rules not processed for user `87e4323175b54c8e9adad6929827c1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:16.539905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f3b5fc67-ef92-4f0e-89dc-266614c22bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:16.569193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:16.743299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 874/1775] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:16.748555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5c6d2-bd69-4a70-998b-51171f3582f7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:16.854032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:16.894343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5c6d2-bd69-4a70-998b-51171f3582f7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:16.903765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1e230-89b0-44b3-b197-b775adf95bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:16.905669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1e230-89b0-44b3-b197-b775adf95bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:16.905978 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1e230-89b0-44b3-b197-b775adf95bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:16.906265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1e230-89b0-44b3-b197-b775adf95bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:16.906986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1e230-89b0-44b3-b197-b775adf95bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:16.915420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1e230-89b0-44b3-b197-b775adf95bd6 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:17.005829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 902/1776] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:16 2022] POST /identity/v3/projects => generated 385 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:17.007362 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5fc67-ef92-4f0e-89dc-266614c22bfd None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:17.030821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5fc67-ef92-4f0e-89dc-266614c22bfd None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:17.181937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651812f8-5fc8-4529-ac97-8afa1bf85098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:17.183440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651812f8-5fc8-4529-ac97-8afa1bf85098 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:17.186812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651812f8-5fc8-4529-ac97-8afa1bf85098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:17.187405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651812f8-5fc8-4529-ac97-8afa1bf85098 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:17.188296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651812f8-5fc8-4529-ac97-8afa1bf85098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:17.194598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651812f8-5fc8-4529-ac97-8afa1bf85098 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:17.242161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 875/1777] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:17 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:17.245992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1e230-89b0-44b3-b197-b775adf95bd6 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:17.267062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1e230-89b0-44b3-b197-b775adf95bd6 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:17.300964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e70bd7-b3f5-43c7-a8f1-9e42b9de1131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:17.302040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e70bd7-b3f5-43c7-a8f1-9e42b9de1131 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:17.302171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e70bd7-b3f5-43c7-a8f1-9e42b9de1131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:17.302378 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e70bd7-b3f5-43c7-a8f1-9e42b9de1131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:17.303126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e70bd7-b3f5-43c7-a8f1-9e42b9de1131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:17.304737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e70bd7-b3f5-43c7-a8f1-9e42b9de1131 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:17.321030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 903/1778] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:17.326183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651812f8-5fc8-4529-ac97-8afa1bf85098 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:17.354817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651812f8-5fc8-4529-ac97-8afa1bf85098 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:17.360687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31707de3-81f2-4a83-94e1-1238f0f46842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:17.361551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31707de3-81f2-4a83-94e1-1238f0f46842 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:17.361796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31707de3-81f2-4a83-94e1-1238f0f46842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:17.362062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31707de3-81f2-4a83-94e1-1238f0f46842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:17.362803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31707de3-81f2-4a83-94e1-1238f0f46842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:17.364461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31707de3-81f2-4a83-94e1-1238f0f46842 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:17.376429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 876/1779] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:17.380395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e70bd7-b3f5-43c7-a8f1-9e42b9de1131 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:17.393774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e70bd7-b3f5-43c7-a8f1-9e42b9de1131 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:17.399898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3a774-a307-4bd7-98fa-f2516d154fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:17.401080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3a774-a307-4bd7-98fa-f2516d154fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:17.401459 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3a774-a307-4bd7-98fa-f2516d154fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:17.401868 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3a774-a307-4bd7-98fa-f2516d154fa3 admin admin] PATH_INFO: `/v3/projects/63a9e0771cc844d9a2eaaf6037c56541/users/f829d63b51ba4b089943b8247b120ec9/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:17.423583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3a774-a307-4bd7-98fa-f2516d154fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a9e0771cc844d9a2eaaf6037c56541, user_id=f829d63b51ba4b089943b8247b120ec9, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:17.426348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3a774-a307-4bd7-98fa-f2516d154fa3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:17.447669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 904/1780] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:17 2022] PUT /identity/v3/projects/63a9e0771cc844d9a2eaaf6037c56541/users/f829d63b51ba4b089943b8247b120ec9/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:17.459827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1373fa-a49c-4ece-87fc-d14ac4c82dae None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:17.461515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1373fa-a49c-4ece-87fc-d14ac4c82dae None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:17.461897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1373fa-a49c-4ece-87fc-d14ac4c82dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:17.497835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-1f1373fa-a49c-4ece-87fc-d14ac4c82dae None None] MFA Rules not processed for user `f829d63b51ba4b089943b8247b120ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:17.499237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1f1373fa-a49c-4ece-87fc-d14ac4c82dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:17.592474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 877/1781] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:17.597684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3a774-a307-4bd7-98fa-f2516d154fa3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:17.704783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3a774-a307-4bd7-98fa-f2516d154fa3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:17.716172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6bcb7-c113-435a-ae65-f043bd435bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:17.717481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6bcb7-c113-435a-ae65-f043bd435bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:17.717796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6bcb7-c113-435a-ae65-f043bd435bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:17.718209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6bcb7-c113-435a-ae65-f043bd435bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:17.719028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6bcb7-c113-435a-ae65-f043bd435bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:17.721753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6bcb7-c113-435a-ae65-f043bd435bc1 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:17.768826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 905/1782] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:17 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:30:17.772464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1373fa-a49c-4ece-87fc-d14ac4c82dae None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:17.789571 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1373fa-a49c-4ece-87fc-d14ac4c82dae None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:17.806127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:18.014646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cb0d9-8789-46bb-a8ea-c4800d15878f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:18.015705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cb0d9-8789-46bb-a8ea-c4800d15878f admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:18.016263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cb0d9-8789-46bb-a8ea-c4800d15878f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:18.016624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cb0d9-8789-46bb-a8ea-c4800d15878f admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:18.017562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cb0d9-8789-46bb-a8ea-c4800d15878f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:18.019583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cb0d9-8789-46bb-a8ea-c4800d15878f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:18.042568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 878/1783] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:17 2022] POST /identity/v3/users => generated 300 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:30:18.046822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6bcb7-c113-435a-ae65-f043bd435bc1 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:18.057895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6bcb7-c113-435a-ae65-f043bd435bc1 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:18.063593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce1521-def9-4dbf-aefd-ee355d0d1fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:18.064481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce1521-def9-4dbf-aefd-ee355d0d1fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:18.064620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce1521-def9-4dbf-aefd-ee355d0d1fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:18.064817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce1521-def9-4dbf-aefd-ee355d0d1fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:18.065759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce1521-def9-4dbf-aefd-ee355d0d1fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:18.067457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce1521-def9-4dbf-aefd-ee355d0d1fbf admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:18.086067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 906/1784] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:18.091366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cb0d9-8789-46bb-a8ea-c4800d15878f admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:18.117611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cb0d9-8789-46bb-a8ea-c4800d15878f admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:18.127734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98954e0a-dc85-4ac1-9483-afee6a26c981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:18.128799 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98954e0a-dc85-4ac1-9483-afee6a26c981 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:18.129386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98954e0a-dc85-4ac1-9483-afee6a26c981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:18.129997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98954e0a-dc85-4ac1-9483-afee6a26c981 admin admin] PATH_INFO: `/v3/projects/562667f789ee46548e2e03f9aebddf8d/users/b5b352ea0f544309a7dc88f0b7d430d6/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:18.190505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98954e0a-dc85-4ac1-9483-afee6a26c981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=562667f789ee46548e2e03f9aebddf8d, user_id=b5b352ea0f544309a7dc88f0b7d430d6, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:18.193625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98954e0a-dc85-4ac1-9483-afee6a26c981 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:18.228448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce1521-def9-4dbf-aefd-ee355d0d1fbf admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:18.228972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 879/1785] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:18 2022] PUT /identity/v3/projects/562667f789ee46548e2e03f9aebddf8d/users/b5b352ea0f544309a7dc88f0b7d430d6/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:18.251416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce1521-def9-4dbf-aefd-ee355d0d1fbf admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:18.441628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb137568-1e87-4d5c-8f8d-be3208054ee3 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:18.441941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb137568-1e87-4d5c-8f8d-be3208054ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:18.442183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb137568-1e87-4d5c-8f8d-be3208054ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:18.486186 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37687e01-6288-4f63-8ad1-c980b009e8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:18.489168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687e01-6288-4f63-8ad1-c980b009e8bb admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:18.489452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687e01-6288-4f63-8ad1-c980b009e8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:18.489735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687e01-6288-4f63-8ad1-c980b009e8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:18.490494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37687e01-6288-4f63-8ad1-c980b009e8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:18.498377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37687e01-6288-4f63-8ad1-c980b009e8bb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:18.499565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-bb137568-1e87-4d5c-8f8d-be3208054ee3 None None] MFA Rules not processed for user `06431315dc8040cbbafbe83a3ed8e0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:18.501712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-bb137568-1e87-4d5c-8f8d-be3208054ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:18.542571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 907/1786] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:18.544130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37687e01-6288-4f63-8ad1-c980b009e8bb admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:18.567712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37687e01-6288-4f63-8ad1-c980b009e8bb admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:18.575454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b80b2-569f-4f83-9666-79862be82955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:18.576667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b80b2-569f-4f83-9666-79862be82955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:18.577349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b80b2-569f-4f83-9666-79862be82955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:18.577848 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b80b2-569f-4f83-9666-79862be82955 admin admin] PATH_INFO: `/v3/projects/562667f789ee46548e2e03f9aebddf8d/users/b5b352ea0f544309a7dc88f0b7d430d6/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:18.596194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b80b2-569f-4f83-9666-79862be82955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=562667f789ee46548e2e03f9aebddf8d, user_id=b5b352ea0f544309a7dc88f0b7d430d6, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:18.599026 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b80b2-569f-4f83-9666-79862be82955 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:18.612591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:18.634479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 908/1787] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:18 2022] PUT /identity/v3/projects/562667f789ee46548e2e03f9aebddf8d/users/b5b352ea0f544309a7dc88f0b7d430d6/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:18.651235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c51d-44e9-4e69-91f4-f09c5aec79fa None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:18.651575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c51d-44e9-4e69-91f4-f09c5aec79fa None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:18.651924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c51d-44e9-4e69-91f4-f09c5aec79fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:18.664900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 880/1788] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:18.744749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-97a9c51d-44e9-4e69-91f4-f09c5aec79fa None None] MFA Rules not processed for user `b5b352ea0f544309a7dc88f0b7d430d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:18.746078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-97a9c51d-44e9-4e69-91f4-f09c5aec79fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:18.900725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 909/1789] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:18.910121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb137568-1e87-4d5c-8f8d-be3208054ee3 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:18.911803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-bb137568-1e87-4d5c-8f8d-be3208054ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:19.033926 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb137568-1e87-4d5c-8f8d-be3208054ee3 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:19.164552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:19.183456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3ed1c-29bc-47e7-9bd8-a3bb5d7c2788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:19.185954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3ed1c-29bc-47e7-9bd8-a3bb5d7c2788 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:19.186350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3ed1c-29bc-47e7-9bd8-a3bb5d7c2788 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:19.186776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3ed1c-29bc-47e7-9bd8-a3bb5d7c2788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:19.207279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3ed1c-29bc-47e7-9bd8-a3bb5d7c2788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:19.210633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3ed1c-29bc-47e7-9bd8-a3bb5d7c2788 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:19.238081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 881/1790] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:19.462780 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe70f14-ec94-4df9-83dd-34e929bfd4f2 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:19.463252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe70f14-ec94-4df9-83dd-34e929bfd4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:19.463343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe70f14-ec94-4df9-83dd-34e929bfd4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:19.516080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ebe70f14-ec94-4df9-83dd-34e929bfd4f2 None None] MFA Rules not processed for user `d0ae34a029c14c6f826456414ac0670d`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:19.517611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ebe70f14-ec94-4df9-83dd-34e929bfd4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:19.685132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 910/1791] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:20.579607 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3ed1c-29bc-47e7-9bd8-a3bb5d7c2788 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:20.594546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3ed1c-29bc-47e7-9bd8-a3bb5d7c2788 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:20.599443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad13c0-5c88-4b49-97b8-8622d0d5eaef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:20.600153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad13c0-5c88-4b49-97b8-8622d0d5eaef service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:20.600438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad13c0-5c88-4b49-97b8-8622d0d5eaef service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:20.600803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad13c0-5c88-4b49-97b8-8622d0d5eaef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:20.757376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad13c0-5c88-4b49-97b8-8622d0d5eaef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:20.758903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad13c0-5c88-4b49-97b8-8622d0d5eaef service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:20.790110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 882/1792] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:20.866440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe70f14-ec94-4df9-83dd-34e929bfd4f2 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:20.869007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ebe70f14-ec94-4df9-83dd-34e929bfd4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:21.204794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe70f14-ec94-4df9-83dd-34e929bfd4f2 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:21.222986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:21.225307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:21.225530 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:21.229080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] PATH_INFO: `/v3/users/f95b81c7059c4159a65b040f3015344b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:21.241388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f95b81c7059c4159a65b040f3015344b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:21.252080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:21.350510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:21.392046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f95b81c7059c4159a65b040f3015344b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:21.431974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f95b81c7059c4159a65b040f3015344b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:21.455343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f95b81c7059c4159a65b040f3015344b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:21.456285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f95b81c7059c4159a65b040f3015344b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:21.456984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f95b81c7059c4159a65b040f3015344b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:21.492460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 911/1793] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:20 2022] DELETE /identity/v3/users/f95b81c7059c4159a65b040f3015344b => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:21.497825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07df0ae-52e0-461d-82c4-f054e3c1e2cb None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:21.498739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07df0ae-52e0-461d-82c4-f054e3c1e2cb None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:21.499044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07df0ae-52e0-461d-82c4-f054e3c1e2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:21.581054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b07df0ae-52e0-461d-82c4-f054e3c1e2cb None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:21.592253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b07df0ae-52e0-461d-82c4-f054e3c1e2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:21.889993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 883/1794] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:21.917790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:21.920590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:22.099945 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fce03-d55c-41b6-87f1-d114be43f594 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:22.104953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1a396-9ce5-4615-a62d-abf6a71203a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:22.106682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1a396-9ce5-4615-a62d-abf6a71203a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:22.107153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1a396-9ce5-4615-a62d-abf6a71203a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:22.107633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1a396-9ce5-4615-a62d-abf6a71203a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:22.122615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1a396-9ce5-4615-a62d-abf6a71203a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:22.125977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1a396-9ce5-4615-a62d-abf6a71203a5 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:22.145096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 912/1795] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:23.974410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07df0ae-52e0-461d-82c4-f054e3c1e2cb None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:23.975967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b07df0ae-52e0-461d-82c4-f054e3c1e2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:24.000979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07df0ae-52e0-461d-82c4-f054e3c1e2cb None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:24.017435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:24.018498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:24.019166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:24.019550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] PATH_INFO: `/v3/projects/f721837584ae40a99b0415e997f5a44f` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:24.021134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f721837584ae40a99b0415e997f5a44f)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:24.023246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:24.176051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f721837584ae40a99b0415e997f5a44f', 'request_id': 'req-c69ae410-d8b8-4e98-ab8e-50523dc2d164'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.186135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f721837584ae40a99b0415e997f5a44f', 'request_id': 'req-c69ae410-d8b8-4e98-ab8e-50523dc2d164'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.201599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f721837584ae40a99b0415e997f5a44f', 'request_id': 'req-c69ae410-d8b8-4e98-ab8e-50523dc2d164'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.219415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] The token cache is being invalidate because project f721837584ae40a99b0415e997f5a44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:24.219864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.221532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c69ae410-d8b8-4e98-ab8e-50523dc2d164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.224254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 884/1796] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:23 2022] DELETE /identity/v3/projects/f721837584ae40a99b0415e997f5a44f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:24.228255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1a396-9ce5-4615-a62d-abf6a71203a5 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:24.242070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1a396-9ce5-4615-a62d-abf6a71203a5 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:24.324689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008ecf-0ad1-4973-ad5b-13d68804d24f None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:24.325311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008ecf-0ad1-4973-ad5b-13d68804d24f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:24.326032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008ecf-0ad1-4973-ad5b-13d68804d24f None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:24.328537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 885/1797] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:30:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:30:24.353778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:24.357651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:24.358638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:24.359234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] PATH_INFO: `/v3/users/1c2abbfb0fdd4922a17cd47d0765f29a` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:24.362947 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c2abbfb0fdd4922a17cd47d0765f29a)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:24.368273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:24.390150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dd8de-16d7-4e77-a079-119780b01d9e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:24.390721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dd8de-16d7-4e77-a079-119780b01d9e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:24.391239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dd8de-16d7-4e77-a079-119780b01d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:24.464159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d14dd8de-16d7-4e77-a079-119780b01d9e None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:24.468681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d14dd8de-16d7-4e77-a079-119780b01d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:24.486496 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:24.493298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c2abbfb0fdd4922a17cd47d0765f29a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.498447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:24.518987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c2abbfb0fdd4922a17cd47d0765f29a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.536020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c2abbfb0fdd4922a17cd47d0765f29a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.537133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c2abbfb0fdd4922a17cd47d0765f29a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.537962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c2abbfb0fdd4922a17cd47d0765f29a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:24.569746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 913/1798] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:24 2022] DELETE /identity/v3/users/1c2abbfb0fdd4922a17cd47d0765f29a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:24.669384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 886/1799] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:30:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:25.233308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:25.236282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:25.287798 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14dd8de-16d7-4e77-a079-119780b01d9e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:25.290254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d14dd8de-16d7-4e77-a079-119780b01d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:25.334786 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14dd8de-16d7-4e77-a079-119780b01d9e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:25.346602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd4dc0-8fe8-41f7-a7aa-0de0ffb44214 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:25.352579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:25.353826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:25.354125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:25.354540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] PATH_INFO: `/v3/users/27c9fa59b70b44a9b73a547a8bf44848` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:25.356494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c9fa59b70b44a9b73a547a8bf44848)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:25.358695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:25.457318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c9fa59b70b44a9b73a547a8bf44848'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:25.478872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c9fa59b70b44a9b73a547a8bf44848'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:25.487969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079b663-bca9-4b0c-8c5f-f2f5d0bc9df1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:25.488957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b663-bca9-4b0c-8c5f-f2f5d0bc9df1 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:25.489368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b663-bca9-4b0c-8c5f-f2f5d0bc9df1 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:25.489756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b663-bca9-4b0c-8c5f-f2f5d0bc9df1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:25.491307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c9fa59b70b44a9b73a547a8bf44848'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:25.493148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c9fa59b70b44a9b73a547a8bf44848'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:25.494963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4b78e673-6012-4ddc-a7c0-18f63f20e137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c9fa59b70b44a9b73a547a8bf44848'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:25.505168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079b663-bca9-4b0c-8c5f-f2f5d0bc9df1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:25.507070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079b663-bca9-4b0c-8c5f-f2f5d0bc9df1 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:25.517902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 914/1800] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:25 2022] DELETE /identity/v3/users/27c9fa59b70b44a9b73a547a8bf44848 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:25.522925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd79631-d88c-4b97-945f-f9c9c0684f4e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:25.523755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd79631-d88c-4b97-945f-f9c9c0684f4e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:25.523755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd79631-d88c-4b97-945f-f9c9c0684f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:25.530708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 887/1801] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:25.544999 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-dcd79631-d88c-4b97-945f-f9c9c0684f4e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:25.546848 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-dcd79631-d88c-4b97-945f-f9c9c0684f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:25.612404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:25.648528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 915/1802] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:25.923280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079b663-bca9-4b0c-8c5f-f2f5d0bc9df1 service nova] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:25.925111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8079b663-bca9-4b0c-8c5f-f2f5d0bc9df1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:25.988192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079b663-bca9-4b0c-8c5f-f2f5d0bc9df1 service nova] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:26.047150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:26.057320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211a198-af86-436d-b5a1-418b3cd0e167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:26.057980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211a198-af86-436d-b5a1-418b3cd0e167 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:26.058264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211a198-af86-436d-b5a1-418b3cd0e167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:26.058542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211a198-af86-436d-b5a1-418b3cd0e167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:26.073770 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211a198-af86-436d-b5a1-418b3cd0e167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:26.075436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211a198-af86-436d-b5a1-418b3cd0e167 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:26.102882 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 888/1803] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:26.228666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd79631-d88c-4b97-945f-f9c9c0684f4e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:26.230219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-dcd79631-d88c-4b97-945f-f9c9c0684f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:26.300573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211a198-af86-436d-b5a1-418b3cd0e167 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:26.302130 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b211a198-af86-436d-b5a1-418b3cd0e167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:26.337825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211a198-af86-436d-b5a1-418b3cd0e167 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:26.354812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:26.358852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:26.360523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:26.361371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] PATH_INFO: `/v3/projects/f6967ddf216b4f409cd4fccd3b51e212` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:26.365509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6967ddf216b4f409cd4fccd3b51e212)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:26.369637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd79631-d88c-4b97-945f-f9c9c0684f4e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:26.373334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:26.378303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ab513-bba3-4ad0-b661-485343b50408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:26.379194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ab513-bba3-4ad0-b661-485343b50408 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:26.379632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ab513-bba3-4ad0-b661-485343b50408 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:26.379910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ab513-bba3-4ad0-b661-485343b50408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:26.381834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9b0ab513-bba3-4ad0-b661-485343b50408 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:26.492470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6967ddf216b4f409cd4fccd3b51e212', 'request_id': 'req-ec209704-ce97-42ae-b0a3-e83992a917df'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:26.502186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6967ddf216b4f409cd4fccd3b51e212', 'request_id': 'req-ec209704-ce97-42ae-b0a3-e83992a917df'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:26.513540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6967ddf216b4f409cd4fccd3b51e212', 'request_id': 'req-ec209704-ce97-42ae-b0a3-e83992a917df'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:26.534707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ab513-bba3-4ad0-b661-485343b50408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:26.543964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ab513-bba3-4ad0-b661-485343b50408 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:26.550043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] The token cache is being invalidate because project f6967ddf216b4f409cd4fccd3b51e212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:26.550346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:26.552894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ec209704-ce97-42ae-b0a3-e83992a917df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:26.555927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 889/1804] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:26 2022] DELETE /identity/v3/projects/f6967ddf216b4f409cd4fccd3b51e212 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:26.563287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a16201-7eaf-42a1-83f7-a009d9942722 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:26.563602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a16201-7eaf-42a1-83f7-a009d9942722 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:26.563883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a16201-7eaf-42a1-83f7-a009d9942722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:26.570177 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 916/1805] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:26.638422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d2a16201-7eaf-42a1-83f7-a009d9942722 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:26.650411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d2a16201-7eaf-42a1-83f7-a009d9942722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:26.856176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 890/1806] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:26.867262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3903c6f-75b0-40d8-ae7b-ede68cd6049d None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:26.867948 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3903c6f-75b0-40d8-ae7b-ede68cd6049d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:26.868567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3903c6f-75b0-40d8-ae7b-ede68cd6049d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:26.913508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b3903c6f-75b0-40d8-ae7b-ede68cd6049d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:26.914947 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b3903c6f-75b0-40d8-ae7b-ede68cd6049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:26.927910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 917/1807] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:26.929321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a16201-7eaf-42a1-83f7-a009d9942722 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:26.942427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a16201-7eaf-42a1-83f7-a009d9942722 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:26.958700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82979267-55d2-445a-8480-ac08b21b96ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:26.959677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82979267-55d2-445a-8480-ac08b21b96ab admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:26.959963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82979267-55d2-445a-8480-ac08b21b96ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:26.960413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82979267-55d2-445a-8480-ac08b21b96ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:26.961526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82979267-55d2-445a-8480-ac08b21b96ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:26.961973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82979267-55d2-445a-8480-ac08b21b96ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:26.963971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82979267-55d2-445a-8480-ac08b21b96ab admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:26.988407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 891/1808] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:26.993557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3903c6f-75b0-40d8-ae7b-ede68cd6049d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:27.007533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3903c6f-75b0-40d8-ae7b-ede68cd6049d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:27.018247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5a464-a941-458f-b30e-c340db29e998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:27.018943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5a464-a941-458f-b30e-c340db29e998 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:27.019267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5a464-a941-458f-b30e-c340db29e998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:27.019549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5a464-a941-458f-b30e-c340db29e998 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:27.022332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5a464-a941-458f-b30e-c340db29e998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:27.026671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5a464-a941-458f-b30e-c340db29e998 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:27.089369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 918/1809] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:26 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:27.092225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82979267-55d2-445a-8480-ac08b21b96ab admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:27.118543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82979267-55d2-445a-8480-ac08b21b96ab admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:27.238183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:27.250515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312286b6-0f16-4abf-af2d-e56bd3f8a654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:27.252103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312286b6-0f16-4abf-af2d-e56bd3f8a654 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:27.252517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312286b6-0f16-4abf-af2d-e56bd3f8a654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:27.254035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312286b6-0f16-4abf-af2d-e56bd3f8a654 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:27.254782 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312286b6-0f16-4abf-af2d-e56bd3f8a654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:27.257585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312286b6-0f16-4abf-af2d-e56bd3f8a654 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:27.295526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5a464-a941-458f-b30e-c340db29e998 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:27.301654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 892/1810] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:27 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:30:27.318783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5a464-a941-458f-b30e-c340db29e998 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:27.324491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e9fb9-bfe3-4a65-861b-4f15cc074b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:27.325315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e9fb9-bfe3-4a65-861b-4f15cc074b9a admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:27.325601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e9fb9-bfe3-4a65-861b-4f15cc074b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:27.326051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e9fb9-bfe3-4a65-861b-4f15cc074b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:27.326803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e9fb9-bfe3-4a65-861b-4f15cc074b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:27.328624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e9fb9-bfe3-4a65-861b-4f15cc074b9a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:27.349676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312286b6-0f16-4abf-af2d-e56bd3f8a654 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:27.358049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 919/1811] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:27.363442 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312286b6-0f16-4abf-af2d-e56bd3f8a654 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:27.369767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87613da-256b-436e-9c18-f263fe508c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:27.370815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87613da-256b-436e-9c18-f263fe508c8f admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:27.370815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87613da-256b-436e-9c18-f263fe508c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:27.370978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87613da-256b-436e-9c18-f263fe508c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:27.372445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87613da-256b-436e-9c18-f263fe508c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:27.374465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87613da-256b-436e-9c18-f263fe508c8f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:27.390035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 893/1812] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:27.394132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e9fb9-bfe3-4a65-861b-4f15cc074b9a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:27.418846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e9fb9-bfe3-4a65-861b-4f15cc074b9a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:27.427514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6afc2f-2f8c-401a-86ef-06e7157f591c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:27.428662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6afc2f-2f8c-401a-86ef-06e7157f591c admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:27.429722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6afc2f-2f8c-401a-86ef-06e7157f591c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:27.430898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6afc2f-2f8c-401a-86ef-06e7157f591c admin admin] PATH_INFO: `/v3/projects/e79ab68936b549b1be0b50f1721549fd/users/00df0aeb7c674b8d9a0249224f281a37/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:27.460370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6afc2f-2f8c-401a-86ef-06e7157f591c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e79ab68936b549b1be0b50f1721549fd, user_id=00df0aeb7c674b8d9a0249224f281a37, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:27.467740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6afc2f-2f8c-401a-86ef-06e7157f591c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:27.534627 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 920/1813] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:27 2022] PUT /identity/v3/projects/e79ab68936b549b1be0b50f1721549fd/users/00df0aeb7c674b8d9a0249224f281a37/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:27.534827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32198da6-0941-409d-a490-c8d082db34fd None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:27.534997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32198da6-0941-409d-a490-c8d082db34fd None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:27.535266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32198da6-0941-409d-a490-c8d082db34fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:27.604952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-32198da6-0941-409d-a490-c8d082db34fd None None] MFA Rules not processed for user `00df0aeb7c674b8d9a0249224f281a37`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:27.604952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-32198da6-0941-409d-a490-c8d082db34fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:27.815339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 894/1814] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:27.820326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6afc2f-2f8c-401a-86ef-06e7157f591c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:27.946982 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:27.974464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6afc2f-2f8c-401a-86ef-06e7157f591c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:27.980228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3d7e5-e29c-4818-a9ac-b9549957ea4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:27.981018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3d7e5-e29c-4818-a9ac-b9549957ea4b admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:27.981300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3d7e5-e29c-4818-a9ac-b9549957ea4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:27.981587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3d7e5-e29c-4818-a9ac-b9549957ea4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:27.982308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3d7e5-e29c-4818-a9ac-b9549957ea4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:27.984787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3d7e5-e29c-4818-a9ac-b9549957ea4b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:28.029200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 921/1815] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:27 2022] POST /identity/v3/projects => generated 387 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:28.033127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32198da6-0941-409d-a490-c8d082db34fd None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:28.046708 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d228bc-fba1-48b3-8099-ae3a41dfcddf None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:28.048674 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d228bc-fba1-48b3-8099-ae3a41dfcddf None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:28.050462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d228bc-fba1-48b3-8099-ae3a41dfcddf None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:28.051852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32198da6-0941-409d-a490-c8d082db34fd None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:28.053549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 922/1816] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:30:28 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:30:28.158280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65699a8e-ac53-4949-a115-e3752e680858 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:28.158878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65699a8e-ac53-4949-a115-e3752e680858 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:28.158878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65699a8e-ac53-4949-a115-e3752e680858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:28.197357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-65699a8e-ac53-4949-a115-e3752e680858 None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:28.200897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-65699a8e-ac53-4949-a115-e3752e680858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:28.208310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5e8f6-571a-4d60-8d1c-366b0bcc7d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:28.210603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5e8f6-571a-4d60-8d1c-366b0bcc7d7d admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:28.211803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5e8f6-571a-4d60-8d1c-366b0bcc7d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:28.212169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5e8f6-571a-4d60-8d1c-366b0bcc7d7d admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:28.214055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5e8f6-571a-4d60-8d1c-366b0bcc7d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:28.220292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5e8f6-571a-4d60-8d1c-366b0bcc7d7d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:28.282833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 895/1817] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:28 2022] POST /identity/v3/users => generated 300 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:30:28.283268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5e8f6-571a-4d60-8d1c-366b0bcc7d7d admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:28.288128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5eb5e8f6-571a-4d60-8d1c-366b0bcc7d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:28.318139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5e8f6-571a-4d60-8d1c-366b0bcc7d7d admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:28.335551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:28.337009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:28.337299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:28.337578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] PATH_INFO: `/v3/projects/3ca4934988684a0e809719aab3af2feb` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:28.341567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ca4934988684a0e809719aab3af2feb)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:28.343065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:28.413624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 923/1818] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:30:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:28.415651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65699a8e-ac53-4949-a115-e3752e680858 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:28.438040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65699a8e-ac53-4949-a115-e3752e680858 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:28.476391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:28.479650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ca4934988684a0e809719aab3af2feb', 'request_id': 'req-b91f5c2d-2490-4956-8d55-86fa5023d5df'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.503486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ca4934988684a0e809719aab3af2feb', 'request_id': 'req-b91f5c2d-2490-4956-8d55-86fa5023d5df'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.521019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ca4934988684a0e809719aab3af2feb', 'request_id': 'req-b91f5c2d-2490-4956-8d55-86fa5023d5df'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.546588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] The token cache is being invalidate because project 3ca4934988684a0e809719aab3af2feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:28.547136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.548304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.552047 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 896/1819] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:28 2022] DELETE /identity/v3/projects/3ca4934988684a0e809719aab3af2feb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:28.555697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:28.558236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:28.616211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23018da2-d0b7-4641-bd2d-ba9c6835828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:28.617999 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23018da2-d0b7-4641-bd2d-ba9c6835828f admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:28.618641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23018da2-d0b7-4641-bd2d-ba9c6835828f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:28.621584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23018da2-d0b7-4641-bd2d-ba9c6835828f admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:28.623406 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23018da2-d0b7-4641-bd2d-ba9c6835828f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:28.627430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23018da2-d0b7-4641-bd2d-ba9c6835828f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:28.659203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 924/1820] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:28.664326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23018da2-d0b7-4641-bd2d-ba9c6835828f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:28.670176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-23018da2-d0b7-4641-bd2d-ba9c6835828f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:28.694003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23018da2-d0b7-4641-bd2d-ba9c6835828f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:28.701559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17e308-296c-4eed-8cb9-8f46d5003212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:28.702473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17e308-296c-4eed-8cb9-8f46d5003212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:28.702821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17e308-296c-4eed-8cb9-8f46d5003212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:28.703081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17e308-296c-4eed-8cb9-8f46d5003212 admin admin] PATH_INFO: `/v3/projects/059e936926b945d2a7ad127b60639d00/users/1f2b3919f0924f6aa77838d025e1a6f9/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:28.712136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f5c2d-2490-4956-8d55-86fa5023d5df admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:28.719054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:28.719975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:28.720275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:28.720536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] PATH_INFO: `/v3/users/3384561c68dd44a3bc2bb6377306fce8` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:28.721867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3384561c68dd44a3bc2bb6377306fce8)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:28.735635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:28.743598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17e308-296c-4eed-8cb9-8f46d5003212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=059e936926b945d2a7ad127b60639d00, user_id=1f2b3919f0924f6aa77838d025e1a6f9, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:28.750579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17e308-296c-4eed-8cb9-8f46d5003212 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:28.786441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 925/1821] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:28 2022] PUT /identity/v3/projects/059e936926b945d2a7ad127b60639d00/users/1f2b3919f0924f6aa77838d025e1a6f9/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:28.791495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17e308-296c-4eed-8cb9-8f46d5003212 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:28.859520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:28.862128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3384561c68dd44a3bc2bb6377306fce8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.866191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3384561c68dd44a3bc2bb6377306fce8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.884316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3384561c68dd44a3bc2bb6377306fce8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.885180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3384561c68dd44a3bc2bb6377306fce8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.914187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3384561c68dd44a3bc2bb6377306fce8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:28.927388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17e308-296c-4eed-8cb9-8f46d5003212 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:28.939743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 897/1822] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:28 2022] DELETE /identity/v3/users/3384561c68dd44a3bc2bb6377306fce8 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:29.037364 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1689ac-c79e-4909-9336-a8a18ce7b102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:29.038367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1689ac-c79e-4909-9336-a8a18ce7b102 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:29.038673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1689ac-c79e-4909-9336-a8a18ce7b102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:29.039047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1689ac-c79e-4909-9336-a8a18ce7b102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:29.039904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1689ac-c79e-4909-9336-a8a18ce7b102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:29.041693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1689ac-c79e-4909-9336-a8a18ce7b102 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:29.060236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 926/1823] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:29.061582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:29.063369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:29.080206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c686c6-362a-45c1-8400-bd5b9cdfb211 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:29.093175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0564ff0-0341-4206-8b7c-73799d6a6d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:29.094402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0564ff0-0341-4206-8b7c-73799d6a6d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:29.094974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0564ff0-0341-4206-8b7c-73799d6a6d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:29.095396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0564ff0-0341-4206-8b7c-73799d6a6d3d admin admin] PATH_INFO: `/v3/projects/059e936926b945d2a7ad127b60639d00/users/1f2b3919f0924f6aa77838d025e1a6f9/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:29.098383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0564ff0-0341-4206-8b7c-73799d6a6d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=059e936926b945d2a7ad127b60639d00, user_id=1f2b3919f0924f6aa77838d025e1a6f9, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:29.101448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0564ff0-0341-4206-8b7c-73799d6a6d3d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:29.119125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 898/1824] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:29 2022] PUT /identity/v3/projects/059e936926b945d2a7ad127b60639d00/users/1f2b3919f0924f6aa77838d025e1a6f9/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:29.135316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ec671-44f8-470f-bed2-bffa52e4e966 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:29.135715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ec671-44f8-470f-bed2-bffa52e4e966 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:29.136347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ec671-44f8-470f-bed2-bffa52e4e966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:29.194146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-407ec671-44f8-470f-bed2-bffa52e4e966 None None] MFA Rules not processed for user `1f2b3919f0924f6aa77838d025e1a6f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:29.195758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-407ec671-44f8-470f-bed2-bffa52e4e966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:29.321193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 927/1825] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:29.346227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0564ff0-0341-4206-8b7c-73799d6a6d3d admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:29.350672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c0564ff0-0341-4206-8b7c-73799d6a6d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:29.478863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ec671-44f8-470f-bed2-bffa52e4e966 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:29.480319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-407ec671-44f8-470f-bed2-bffa52e4e966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:29.503713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:29.528054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0564ff0-0341-4206-8b7c-73799d6a6d3d admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:29.538374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81365b-7043-4cc5-8246-d5797b580362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:29.539408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81365b-7043-4cc5-8246-d5797b580362 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:29.540155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81365b-7043-4cc5-8246-d5797b580362 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:29.540246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81365b-7043-4cc5-8246-d5797b580362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:29.569170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81365b-7043-4cc5-8246-d5797b580362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:29.577340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81365b-7043-4cc5-8246-d5797b580362 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:29.623763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 899/1826] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:29.713628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:29.764847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ec671-44f8-470f-bed2-bffa52e4e966 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:29.777098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47ba15-75a4-4f2f-8292-cd2f8109c35b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:29.778055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47ba15-75a4-4f2f-8292-cd2f8109c35b service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:29.778351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47ba15-75a4-4f2f-8292-cd2f8109c35b service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:29.778621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47ba15-75a4-4f2f-8292-cd2f8109c35b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:29.784974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-df47ba15-75a4-4f2f-8292-cd2f8109c35b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:29.804935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47ba15-75a4-4f2f-8292-cd2f8109c35b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:29.806784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47ba15-75a4-4f2f-8292-cd2f8109c35b service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:29.839133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 928/1827] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:29.855152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81365b-7043-4cc5-8246-d5797b580362 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:29.857506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1d81365b-7043-4cc5-8246-d5797b580362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:29.880976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47ba15-75a4-4f2f-8292-cd2f8109c35b service nova] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:29.990313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47ba15-75a4-4f2f-8292-cd2f8109c35b service nova] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:29.991458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81365b-7043-4cc5-8246-d5797b580362 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:29.995687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:29.997914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:29.998689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:29.998809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:30.000524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:30.001172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:30.001779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] PATH_INFO: `/v3/users/87e4323175b54c8e9adad6929827c1f3` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:30.003792 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] PATH_INFO: `/v3/users/00df0aeb7c674b8d9a0249224f281a37` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:30.005057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e4323175b54c8e9adad6929827c1f3)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:30.005375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] RBAC: Authorizing `identity:delete_user(user_id=00df0aeb7c674b8d9a0249224f281a37)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:30.007858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:30.008782 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:30.074818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e4323175b54c8e9adad6929827c1f3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.076326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e4323175b54c8e9adad6929827c1f3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.081709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00df0aeb7c674b8d9a0249224f281a37'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.087986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e4323175b54c8e9adad6929827c1f3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.096452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e4323175b54c8e9adad6929827c1f3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.113503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cbceb76b-eee4-41bc-a673-4c4d5e77d985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e4323175b54c8e9adad6929827c1f3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.124552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00df0aeb7c674b8d9a0249224f281a37'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.137920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00df0aeb7c674b8d9a0249224f281a37'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.139038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00df0aeb7c674b8d9a0249224f281a37'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.139213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c211dc6b-2f91-45c5-9f41-e859c468a73c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00df0aeb7c674b8d9a0249224f281a37'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.141688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 900/1828] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:29 2022] DELETE /identity/v3/users/87e4323175b54c8e9adad6929827c1f3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:30.149067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc81c-8785-42eb-b0ad-58bf299ca2cd None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:30.149360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc81c-8785-42eb-b0ad-58bf299ca2cd None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:30.149543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc81c-8785-42eb-b0ad-58bf299ca2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:30.160972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 929/1829] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:29 2022] DELETE /identity/v3/users/00df0aeb7c674b8d9a0249224f281a37 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:30.165462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86071a-86cf-4db5-abaf-572c0318596d None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:30.165890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86071a-86cf-4db5-abaf-572c0318596d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:30.166326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86071a-86cf-4db5-abaf-572c0318596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:30.189794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-000dc81c-8785-42eb-b0ad-58bf299ca2cd None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:30.195610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-2f86071a-86cf-4db5-abaf-572c0318596d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:30.196787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2f86071a-86cf-4db5-abaf-572c0318596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:30.198745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-000dc81c-8785-42eb-b0ad-58bf299ca2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:30.350358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:30.388670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 930/1830] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:30.388670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86071a-86cf-4db5-abaf-572c0318596d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:30.391641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 901/1831] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:30.392071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2f86071a-86cf-4db5-abaf-572c0318596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:30.417292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86071a-86cf-4db5-abaf-572c0318596d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:30.434676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc81c-8785-42eb-b0ad-58bf299ca2cd None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:30.437708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-000dc81c-8785-42eb-b0ad-58bf299ca2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:30.440124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:30.442246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:30.442835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:30.443785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] PATH_INFO: `/v3/projects/ae8e79e46ed4453e8ee3a16498afe044` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:30.445625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae8e79e46ed4453e8ee3a16498afe044)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:30.451236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:30.452794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:30.573182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc81c-8785-42eb-b0ad-58bf299ca2cd None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:30.627126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae8e79e46ed4453e8ee3a16498afe044', 'request_id': 'req-a96a63dc-510c-43ea-bb37-a331be480194'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.644697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae8e79e46ed4453e8ee3a16498afe044', 'request_id': 'req-a96a63dc-510c-43ea-bb37-a331be480194'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.670321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae8e79e46ed4453e8ee3a16498afe044', 'request_id': 'req-a96a63dc-510c-43ea-bb37-a331be480194'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.724248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] The token cache is being invalidate because project ae8e79e46ed4453e8ee3a16498afe044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:30.724624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.726089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:30.729286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 931/1832] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:30 2022] DELETE /identity/v3/projects/ae8e79e46ed4453e8ee3a16498afe044 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:30.733029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:30.739379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:30.771789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006c0f5-41d8-4153-a195-0ff218de53c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:30.772465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006c0f5-41d8-4153-a195-0ff218de53c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:30.772747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006c0f5-41d8-4153-a195-0ff218de53c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:30.773053 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006c0f5-41d8-4153-a195-0ff218de53c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:30.781360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e006c0f5-41d8-4153-a195-0ff218de53c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:30.895309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a63dc-510c-43ea-bb37-a331be480194 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:30.900104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13edbca-59ac-488b-922b-fc267988a30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:30.900925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13edbca-59ac-488b-922b-fc267988a30b service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:30.901231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13edbca-59ac-488b-922b-fc267988a30b service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:30.901562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13edbca-59ac-488b-922b-fc267988a30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:30.904120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e13edbca-59ac-488b-922b-fc267988a30b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:30.917244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006c0f5-41d8-4153-a195-0ff218de53c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:30.919946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006c0f5-41d8-4153-a195-0ff218de53c7 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:30.927778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13edbca-59ac-488b-922b-fc267988a30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:30.932769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13edbca-59ac-488b-922b-fc267988a30b service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:30.937539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 902/1833] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:30.939640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc359a6-448d-4729-8f94-9be0983e0330 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:30.940155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc359a6-448d-4729-8f94-9be0983e0330 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:30.940673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc359a6-448d-4729-8f94-9be0983e0330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:30.964898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-3cc359a6-448d-4729-8f94-9be0983e0330 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:30.966796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3cc359a6-448d-4729-8f94-9be0983e0330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:30.972914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 932/1834] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:30.992706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 903/1835] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:30.995363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ddf02-d8f9-4eaa-b7b5-4ca162a41e37 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:30.995660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ddf02-d8f9-4eaa-b7b5-4ca162a41e37 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:30.996009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ddf02-d8f9-4eaa-b7b5-4ca162a41e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:31.073621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f47ddf02-d8f9-4eaa-b7b5-4ca162a41e37 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:31.074883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f47ddf02-d8f9-4eaa-b7b5-4ca162a41e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:31.106514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 933/1836] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:31.108306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc359a6-448d-4729-8f94-9be0983e0330 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:31.129090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc359a6-448d-4729-8f94-9be0983e0330 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:31.140222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9abc-1b1a-4788-bf36-466e2d6ac59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:31.142365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9abc-1b1a-4788-bf36-466e2d6ac59b admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:31.142936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9abc-1b1a-4788-bf36-466e2d6ac59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:31.143898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9abc-1b1a-4788-bf36-466e2d6ac59b admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:31.145184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9abc-1b1a-4788-bf36-466e2d6ac59b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:31.145838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9abc-1b1a-4788-bf36-466e2d6ac59b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:31.148243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9abc-1b1a-4788-bf36-466e2d6ac59b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:31.209841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 904/1837] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:31.210760 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ddf02-d8f9-4eaa-b7b5-4ca162a41e37 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:31.236097 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ddf02-d8f9-4eaa-b7b5-4ca162a41e37 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:31.277762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c14a2a-d78a-4635-bb43-e3bcaa0a9d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:31.278511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c14a2a-d78a-4635-bb43-e3bcaa0a9d20 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:31.278731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c14a2a-d78a-4635-bb43-e3bcaa0a9d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:31.279008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c14a2a-d78a-4635-bb43-e3bcaa0a9d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:31.279721 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c14a2a-d78a-4635-bb43-e3bcaa0a9d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:31.297316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c14a2a-d78a-4635-bb43-e3bcaa0a9d20 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:31.425302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 934/1838] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:31 2022] POST /identity/v3/projects => generated 363 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:31.431087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9abc-1b1a-4788-bf36-466e2d6ac59b admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:31.448918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9abc-1b1a-4788-bf36-466e2d6ac59b admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:31.515587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:31.529500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58d05f-d465-442d-8467-26b2bf982942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:31.530206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58d05f-d465-442d-8467-26b2bf982942 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:31.530590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58d05f-d465-442d-8467-26b2bf982942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:31.530939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58d05f-d465-442d-8467-26b2bf982942 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:31.531715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58d05f-d465-442d-8467-26b2bf982942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:31.533800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58d05f-d465-442d-8467-26b2bf982942 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:31.560984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 905/1839] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:31 2022] POST /identity/v3/users => generated 282 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:31.565228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c14a2a-d78a-4635-bb43-e3bcaa0a9d20 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:31.580153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c14a2a-d78a-4635-bb43-e3bcaa0a9d20 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:31.599531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5581d5-bae7-46fd-8ebc-28ce531db612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:31.600226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5581d5-bae7-46fd-8ebc-28ce531db612 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:31.600505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5581d5-bae7-46fd-8ebc-28ce531db612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:31.600781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5581d5-bae7-46fd-8ebc-28ce531db612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:31.601560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5581d5-bae7-46fd-8ebc-28ce531db612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:31.603163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5581d5-bae7-46fd-8ebc-28ce531db612 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:31.629470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 935/1840] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:31.633708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58d05f-d465-442d-8467-26b2bf982942 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:31.648840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58d05f-d465-442d-8467-26b2bf982942 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:31.655641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee1f9c-b2a8-4d52-ab07-885fe2815027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:31.657119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee1f9c-b2a8-4d52-ab07-885fe2815027 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:31.657119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee1f9c-b2a8-4d52-ab07-885fe2815027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:31.657119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee1f9c-b2a8-4d52-ab07-885fe2815027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:31.658447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee1f9c-b2a8-4d52-ab07-885fe2815027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:31.660836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee1f9c-b2a8-4d52-ab07-885fe2815027 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:31.700787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5581d5-bae7-46fd-8ebc-28ce531db612 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:31.701294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 906/1841] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:31.716671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5581d5-bae7-46fd-8ebc-28ce531db612 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:31.729310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1522bc-4913-4965-8bab-e36534da37e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:31.730971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1522bc-4913-4965-8bab-e36534da37e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:31.731396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1522bc-4913-4965-8bab-e36534da37e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:31.731825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1522bc-4913-4965-8bab-e36534da37e8 admin admin] PATH_INFO: `/v3/projects/80efdb219409498f9ad4e35f08790314/users/04b40efa87794c118513a9f72ee0972a/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:31.773266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1522bc-4913-4965-8bab-e36534da37e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80efdb219409498f9ad4e35f08790314, user_id=04b40efa87794c118513a9f72ee0972a, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:31.776429 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1522bc-4913-4965-8bab-e36534da37e8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:31.804987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 936/1842] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:31 2022] PUT /identity/v3/projects/80efdb219409498f9ad4e35f08790314/users/04b40efa87794c118513a9f72ee0972a/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:31.806931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b115247-7973-48a2-a466-45d5fa559ced None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:31.807409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b115247-7973-48a2-a466-45d5fa559ced None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:31.807823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b115247-7973-48a2-a466-45d5fa559ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:31.871073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-1b115247-7973-48a2-a466-45d5fa559ced None None] MFA Rules not processed for user `04b40efa87794c118513a9f72ee0972a`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:31.872449 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1b115247-7973-48a2-a466-45d5fa559ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:32.006099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 907/1843] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:32.010341 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1522bc-4913-4965-8bab-e36534da37e8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:32.059609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:32.127980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1522bc-4913-4965-8bab-e36534da37e8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:32.144518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff79f8b-2cac-49ed-8e35-cff6c91f22f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:32.147095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff79f8b-2cac-49ed-8e35-cff6c91f22f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:32.147634 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff79f8b-2cac-49ed-8e35-cff6c91f22f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:32.148149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff79f8b-2cac-49ed-8e35-cff6c91f22f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:32.149586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff79f8b-2cac-49ed-8e35-cff6c91f22f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:32.152529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff79f8b-2cac-49ed-8e35-cff6c91f22f2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:32.204521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 937/1844] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:32 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:32.208916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b115247-7973-48a2-a466-45d5fa559ced None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:32.234129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b115247-7973-48a2-a466-45d5fa559ced None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:32.351637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1121a6-5553-49e9-821e-550669784220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:32.352550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1121a6-5553-49e9-821e-550669784220 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:32.352829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1121a6-5553-49e9-821e-550669784220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:32.353138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1121a6-5553-49e9-821e-550669784220 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:32.353884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1121a6-5553-49e9-821e-550669784220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:32.355977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1121a6-5553-49e9-821e-550669784220 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:32.384701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 908/1845] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:32 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:32.385653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff79f8b-2cac-49ed-8e35-cff6c91f22f2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:32.418944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff79f8b-2cac-49ed-8e35-cff6c91f22f2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:32.428098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e0144-25a6-4fae-8448-719263fc78c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:32.428812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0144-25a6-4fae-8448-719263fc78c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:32.430437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0144-25a6-4fae-8448-719263fc78c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:32.430732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0144-25a6-4fae-8448-719263fc78c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:32.433243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e0144-25a6-4fae-8448-719263fc78c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:32.436261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e0144-25a6-4fae-8448-719263fc78c4 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:32.480023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 938/1846] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:32.485085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1121a6-5553-49e9-821e-550669784220 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:32.495663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1121a6-5553-49e9-821e-550669784220 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:32.518921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111f1fc-1e16-44e3-b857-c284e4e36d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:32.519918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111f1fc-1e16-44e3-b857-c284e4e36d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:32.520288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111f1fc-1e16-44e3-b857-c284e4e36d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:32.520638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111f1fc-1e16-44e3-b857-c284e4e36d8c admin admin] PATH_INFO: `/v3/projects/1f57aca0025b4ee19fd4f70a5cd77dc3/users/4c3d401a5b4d4aeb99573863be53e385/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:32.552113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111f1fc-1e16-44e3-b857-c284e4e36d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f57aca0025b4ee19fd4f70a5cd77dc3, user_id=4c3d401a5b4d4aeb99573863be53e385, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:32.556399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111f1fc-1e16-44e3-b857-c284e4e36d8c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:32.583886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 909/1847] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:32 2022] PUT /identity/v3/projects/1f57aca0025b4ee19fd4f70a5cd77dc3/users/4c3d401a5b4d4aeb99573863be53e385/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:32.591515 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e0144-25a6-4fae-8448-719263fc78c4 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:32.612370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e0144-25a6-4fae-8448-719263fc78c4 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:32.738859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e31c69-0b40-406b-86b9-0c2893be814b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:32.739767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31c69-0b40-406b-86b9-0c2893be814b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:32.740051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31c69-0b40-406b-86b9-0c2893be814b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:32.740334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31c69-0b40-406b-86b9-0c2893be814b admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:32.741192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e31c69-0b40-406b-86b9-0c2893be814b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:32.742844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e31c69-0b40-406b-86b9-0c2893be814b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:32.770307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111f1fc-1e16-44e3-b857-c284e4e36d8c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:32.772763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 939/1848] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:32.787654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111f1fc-1e16-44e3-b857-c284e4e36d8c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:32.814137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b1907-6e03-40db-b1de-d55d0fc9ee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:32.815709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b1907-6e03-40db-b1de-d55d0fc9ee01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:32.816166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b1907-6e03-40db-b1de-d55d0fc9ee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:32.816275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b1907-6e03-40db-b1de-d55d0fc9ee01 admin admin] PATH_INFO: `/v3/projects/1f57aca0025b4ee19fd4f70a5cd77dc3/users/4c3d401a5b4d4aeb99573863be53e385/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:32.820985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b1907-6e03-40db-b1de-d55d0fc9ee01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f57aca0025b4ee19fd4f70a5cd77dc3, user_id=4c3d401a5b4d4aeb99573863be53e385, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:32.829879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b1907-6e03-40db-b1de-d55d0fc9ee01 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:32.871168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 910/1849] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:32 2022] PUT /identity/v3/projects/1f57aca0025b4ee19fd4f70a5cd77dc3/users/4c3d401a5b4d4aeb99573863be53e385/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:32.882500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afa7c1-20eb-464f-a9f5-7a6bc640c91e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:32.882915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afa7c1-20eb-464f-a9f5-7a6bc640c91e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:32.883066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afa7c1-20eb-464f-a9f5-7a6bc640c91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:32.937048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-71afa7c1-20eb-464f-a9f5-7a6bc640c91e None None] MFA Rules not processed for user `4c3d401a5b4d4aeb99573863be53e385`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:32.938287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-71afa7c1-20eb-464f-a9f5-7a6bc640c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:33.097465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b1907-6e03-40db-b1de-d55d0fc9ee01 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:33.099723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-411b1907-6e03-40db-b1de-d55d0fc9ee01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:33.130175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:33.207220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 940/1850] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:33.218935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afa7c1-20eb-464f-a9f5-7a6bc640c91e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:33.221090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-71afa7c1-20eb-464f-a9f5-7a6bc640c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:33.227444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b1907-6e03-40db-b1de-d55d0fc9ee01 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:33.237747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:33.239028 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:33.239588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:33.240065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] PATH_INFO: `/v3/projects/42ec461cb12e4710a19d58ae6fc4fbc3` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:33.241886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=42ec461cb12e4710a19d58ae6fc4fbc3)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:33.244273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:33.271199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:33.329020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ec461cb12e4710a19d58ae6fc4fbc3', 'request_id': 'req-5eeab09e-84e7-4052-9ac1-5df7c85328ad'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.344540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ec461cb12e4710a19d58ae6fc4fbc3', 'request_id': 'req-5eeab09e-84e7-4052-9ac1-5df7c85328ad'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.353477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afa7c1-20eb-464f-a9f5-7a6bc640c91e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:33.356579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ec461cb12e4710a19d58ae6fc4fbc3', 'request_id': 'req-5eeab09e-84e7-4052-9ac1-5df7c85328ad'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.377950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] The token cache is being invalidate because project 42ec461cb12e4710a19d58ae6fc4fbc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:33.378708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.379542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.387020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 911/1851] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:33 2022] DELETE /identity/v3/projects/42ec461cb12e4710a19d58ae6fc4fbc3 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:33.391526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:33.394156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:33.477123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:33.480135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:33.481380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:33.483632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:33.486655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:33.548370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeab09e-84e7-4052-9ac1-5df7c85328ad admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:33.561308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:33.562221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:33.562505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:33.562778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] PATH_INFO: `/v3/users/f829d63b51ba4b089943b8247b120ec9` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:33.566372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f829d63b51ba4b089943b8247b120ec9)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:33.569681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:33.671786 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:33.675268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f829d63b51ba4b089943b8247b120ec9'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.676457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f829d63b51ba4b089943b8247b120ec9'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.685932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:33.693620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f829d63b51ba4b089943b8247b120ec9'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.693843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:33.695309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f829d63b51ba4b089943b8247b120ec9'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.708802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 941/1852] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:33.709586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:33.712219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:33.737664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f829d63b51ba4b089943b8247b120ec9'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.739431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736598e-7043-4ba6-8b7c-271acfff2293 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:33.750374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:33.751587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:33.751979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:33.752418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] PATH_INFO: `/v3/projects/e79ab68936b549b1be0b50f1721549fd` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:33.756039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e79ab68936b549b1be0b50f1721549fd)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:33.759464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:33.774517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 912/1853] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:33 2022] DELETE /identity/v3/users/f829d63b51ba4b089943b8247b120ec9 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:33.899507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:33.901078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:33.901757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e79ab68936b549b1be0b50f1721549fd', 'request_id': 'req-977ce65a-63f4-467d-8c09-35427b0a63da'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.917371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e79ab68936b549b1be0b50f1721549fd', 'request_id': 'req-977ce65a-63f4-467d-8c09-35427b0a63da'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.937277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e79ab68936b549b1be0b50f1721549fd', 'request_id': 'req-977ce65a-63f4-467d-8c09-35427b0a63da'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.987649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] The token cache is being invalidate because project e79ab68936b549b1be0b50f1721549fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:33.987970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.991723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:33.995027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 942/1854] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:33 2022] DELETE /identity/v3/projects/e79ab68936b549b1be0b50f1721549fd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:34.000960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:34.006012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:34.032833 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:34.070383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e7271-a2e9-4fdb-ab03-c93f3ccc72a7 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:34.133575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ce65a-63f4-467d-8c09-35427b0a63da admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:34.143913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:34.145944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:34.146366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:34.146782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] PATH_INFO: `/v3/users/1f2b3919f0924f6aa77838d025e1a6f9` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:34.148434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2b3919f0924f6aa77838d025e1a6f9)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:34.151541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:34.238014 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b876c51-370c-4c89-866a-53f2668e7efd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:34.240134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b876c51-370c-4c89-866a-53f2668e7efd service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:34.241771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b876c51-370c-4c89-866a-53f2668e7efd service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:34.242286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b876c51-370c-4c89-866a-53f2668e7efd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:34.249008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2b3919f0924f6aa77838d025e1a6f9'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:34.252076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7b876c51-370c-4c89-866a-53f2668e7efd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:34.303046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2b3919f0924f6aa77838d025e1a6f9'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:34.322352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2b3919f0924f6aa77838d025e1a6f9'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:34.323556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2b3919f0924f6aa77838d025e1a6f9'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:34.324162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2b3919f0924f6aa77838d025e1a6f9'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:34.350826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 943/1855] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:33 2022] DELETE /identity/v3/users/1f2b3919f0924f6aa77838d025e1a6f9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:34.395698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b876c51-370c-4c89-866a-53f2668e7efd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:34.397306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b876c51-370c-4c89-866a-53f2668e7efd service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:34.429953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 913/1856] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:34.841730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:34.843334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:34.879172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b876c51-370c-4c89-866a-53f2668e7efd service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:34.911004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b876c51-370c-4c89-866a-53f2668e7efd service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:34.954452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:34.985475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9176fa-6311-46a0-8909-b35fcccf87fd admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:34.991259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:34.992409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:34.992695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:34.993016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] PATH_INFO: `/v3/projects/63a9e0771cc844d9a2eaaf6037c56541` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:34.993652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:34.994448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a9e0771cc844d9a2eaaf6037c56541)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:34.994555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:34.994827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:34.995100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] PATH_INFO: `/v3/projects/059e936926b945d2a7ad127b60639d00` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:34.996849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059e936926b945d2a7ad127b60639d00)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:34.997064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:34.999243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:35.032712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:35.051382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a9e0771cc844d9a2eaaf6037c56541', 'request_id': 'req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.060210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059e936926b945d2a7ad127b60639d00', 'request_id': 'req-bcbbf5d6-578f-478a-9ef1-24ee08055092'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.065829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a9e0771cc844d9a2eaaf6037c56541', 'request_id': 'req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.066830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059e936926b945d2a7ad127b60639d00', 'request_id': 'req-bcbbf5d6-578f-478a-9ef1-24ee08055092'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.073206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a9e0771cc844d9a2eaaf6037c56541', 'request_id': 'req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.077337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059e936926b945d2a7ad127b60639d00', 'request_id': 'req-bcbbf5d6-578f-478a-9ef1-24ee08055092'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.092019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] The token cache is being invalidate because project 63a9e0771cc844d9a2eaaf6037c56541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:35.092395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.093020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.096927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 944/1857] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:34 2022] DELETE /identity/v3/projects/63a9e0771cc844d9a2eaaf6037c56541 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:35.097065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] The token cache is being invalidate because project 059e936926b945d2a7ad127b60639d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:35.097065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.097065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bcbbf5d6-578f-478a-9ef1-24ee08055092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.097497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 914/1858] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:34 2022] DELETE /identity/v3/projects/059e936926b945d2a7ad127b60639d00 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:35.098666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:35.100109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:35.102326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0c8fd-c6fb-4fab-b342-2ceb81313350 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.103411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0c8fd-c6fb-4fab-b342-2ceb81313350 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.104000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0c8fd-c6fb-4fab-b342-2ceb81313350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.125453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-85c0c8fd-c6fb-4fab-b342-2ceb81313350 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:35.126742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-85c0c8fd-c6fb-4fab-b342-2ceb81313350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:35.204584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d12038-cd34-4b99-b9fd-1dc11d0c4d4e admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:35.234080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 915/1859] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:35.242262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e5e00-d87e-46f7-85d2-da7eea1528b2 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.242546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e5e00-d87e-46f7-85d2-da7eea1528b2 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.242822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e5e00-d87e-46f7-85d2-da7eea1528b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.275549 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-af3e5e00-d87e-46f7-85d2-da7eea1528b2 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:35.276896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-af3e5e00-d87e-46f7-85d2-da7eea1528b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:35.286710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 916/1860] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:35.288129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:35.289198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.289556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.289844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] PATH_INFO: `/v3/users/b5b352ea0f544309a7dc88f0b7d430d6` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.290598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e5e00-d87e-46f7-85d2-da7eea1528b2 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:35.291059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5b352ea0f544309a7dc88f0b7d430d6)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:35.293112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:35.304056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e5e00-d87e-46f7-85d2-da7eea1528b2 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:35.310033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9be7e-a896-4030-96d3-9b038bd6a426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:35.311031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9be7e-a896-4030-96d3-9b038bd6a426 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.311324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9be7e-a896-4030-96d3-9b038bd6a426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.311717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9be7e-a896-4030-96d3-9b038bd6a426 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.312345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9be7e-a896-4030-96d3-9b038bd6a426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:35.312653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9be7e-a896-4030-96d3-9b038bd6a426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:35.315138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9be7e-a896-4030-96d3-9b038bd6a426 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:35.331817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 917/1861] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:30:35.334972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9be7e-a896-4030-96d3-9b038bd6a426 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:35.350216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9be7e-a896-4030-96d3-9b038bd6a426 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:35.355055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9131995-95db-49f1-8b54-44dd38fd7896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:35.355542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5b352ea0f544309a7dc88f0b7d430d6'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.355854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9131995-95db-49f1-8b54-44dd38fd7896 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.356305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9131995-95db-49f1-8b54-44dd38fd7896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.356618 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9131995-95db-49f1-8b54-44dd38fd7896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.357349 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9131995-95db-49f1-8b54-44dd38fd7896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:35.359294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9131995-95db-49f1-8b54-44dd38fd7896 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:35.371578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5b352ea0f544309a7dc88f0b7d430d6'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.373311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:35.382311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5b352ea0f544309a7dc88f0b7d430d6'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.383307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5b352ea0f544309a7dc88f0b7d430d6'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.383947 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5b352ea0f544309a7dc88f0b7d430d6'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:35.400184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 918/1862] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:35 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:30:35.401112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9131995-95db-49f1-8b54-44dd38fd7896 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:35.403315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a9131995-95db-49f1-8b54-44dd38fd7896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:35.406641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 945/1863] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:35 2022] DELETE /identity/v3/users/b5b352ea0f544309a7dc88f0b7d430d6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:35.445339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:35.518909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9131995-95db-49f1-8b54-44dd38fd7896 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:35.611074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44ecd7-8944-4b3d-a188-5d2c884857e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:35.612293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44ecd7-8944-4b3d-a188-5d2c884857e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.612654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44ecd7-8944-4b3d-a188-5d2c884857e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.613077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44ecd7-8944-4b3d-a188-5d2c884857e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.614217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44ecd7-8944-4b3d-a188-5d2c884857e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:35.617348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44ecd7-8944-4b3d-a188-5d2c884857e8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:35.650420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 919/1864] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:35 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:35.657568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:35.674539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c36eb5-fc24-45a4-8843-8ca662f9db5f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:35.689688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd1515-973c-4c09-bc02-65a2ea9db038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:35.690554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd1515-973c-4c09-bc02-65a2ea9db038 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.690939 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd1515-973c-4c09-bc02-65a2ea9db038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.691320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd1515-973c-4c09-bc02-65a2ea9db038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.692195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd1515-973c-4c09-bc02-65a2ea9db038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:35.701917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd1515-973c-4c09-bc02-65a2ea9db038 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:35.734003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 946/1865] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:35.737992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44ecd7-8944-4b3d-a188-5d2c884857e8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:35.751992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44ecd7-8944-4b3d-a188-5d2c884857e8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:35.765495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3bd36-9f7e-4c3d-bddc-5b6a0c1df4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:35.766690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3bd36-9f7e-4c3d-bddc-5b6a0c1df4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.767127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3bd36-9f7e-4c3d-bddc-5b6a0c1df4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.767980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3bd36-9f7e-4c3d-bddc-5b6a0c1df4a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.768908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3bd36-9f7e-4c3d-bddc-5b6a0c1df4a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:35.773832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3bd36-9f7e-4c3d-bddc-5b6a0c1df4a1 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:35.804067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 920/1866] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:35.806305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd1515-973c-4c09-bc02-65a2ea9db038 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:35.821405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd1515-973c-4c09-bc02-65a2ea9db038 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:35.836049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fed76b-a4ca-4a98-adc3-fc4315637133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:35.836997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fed76b-a4ca-4a98-adc3-fc4315637133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.837287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fed76b-a4ca-4a98-adc3-fc4315637133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.837571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fed76b-a4ca-4a98-adc3-fc4315637133 admin admin] PATH_INFO: `/v3/projects/d112fea2e24f469a8d54a7950aa74862/users/767b5629a2ce472cb57ebc2421597338/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.875731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fed76b-a4ca-4a98-adc3-fc4315637133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d112fea2e24f469a8d54a7950aa74862, user_id=767b5629a2ce472cb57ebc2421597338, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:35.878531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fed76b-a4ca-4a98-adc3-fc4315637133 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:35.911556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 947/1867] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:35 2022] PUT /identity/v3/projects/d112fea2e24f469a8d54a7950aa74862/users/767b5629a2ce472cb57ebc2421597338/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:35.915623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d9380-ba91-4f1e-b493-931368c6eea6 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:35.916114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d9380-ba91-4f1e-b493-931368c6eea6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:35.916403 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d9380-ba91-4f1e-b493-931368c6eea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:35.967359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fed76b-a4ca-4a98-adc3-fc4315637133 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:35.969857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-72fed76b-a4ca-4a98-adc3-fc4315637133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:35.974485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-5f6d9380-ba91-4f1e-b493-931368c6eea6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:35.977077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5f6d9380-ba91-4f1e-b493-931368c6eea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:36.130711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fed76b-a4ca-4a98-adc3-fc4315637133 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:36.147593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:36.148349 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 921/1868] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:36.150076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:36.150687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:36.151724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] PATH_INFO: `/v3/projects/562667f789ee46548e2e03f9aebddf8d` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:36.158020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] RBAC: Authorizing `identity:delete_project(project_id=562667f789ee46548e2e03f9aebddf8d)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:36.160586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:36.162539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d9380-ba91-4f1e-b493-931368c6eea6 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:36.163981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5f6d9380-ba91-4f1e-b493-931368c6eea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:36.242903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:36.257821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d9380-ba91-4f1e-b493-931368c6eea6 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:36.288921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '562667f789ee46548e2e03f9aebddf8d', 'request_id': 'req-a4059905-ffca-45f5-afac-c942dffba51a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:36.305360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '562667f789ee46548e2e03f9aebddf8d', 'request_id': 'req-a4059905-ffca-45f5-afac-c942dffba51a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:36.320082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '562667f789ee46548e2e03f9aebddf8d', 'request_id': 'req-a4059905-ffca-45f5-afac-c942dffba51a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:36.355169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] The token cache is being invalidate because project 562667f789ee46548e2e03f9aebddf8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:36.355507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:36.356182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a4059905-ffca-45f5-afac-c942dffba51a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:36.364245 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 948/1869] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:35 2022] DELETE /identity/v3/projects/562667f789ee46548e2e03f9aebddf8d => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:36.369312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895765e6-5cd1-4dd7-8db6-8f27021e462f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:36.373168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895765e6-5cd1-4dd7-8db6-8f27021e462f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:36.373458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895765e6-5cd1-4dd7-8db6-8f27021e462f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:36.374691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887be4e9-aa7c-4179-89ab-732c11145e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:36.378474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887be4e9-aa7c-4179-89ab-732c11145e45 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:36.378474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887be4e9-aa7c-4179-89ab-732c11145e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:36.378474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887be4e9-aa7c-4179-89ab-732c11145e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:36.382247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-887be4e9-aa7c-4179-89ab-732c11145e45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:36.441348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-895765e6-5cd1-4dd7-8db6-8f27021e462f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:36.442499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-895765e6-5cd1-4dd7-8db6-8f27021e462f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:36.504154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:36.543695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887be4e9-aa7c-4179-89ab-732c11145e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:36.547229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887be4e9-aa7c-4179-89ab-732c11145e45 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:36.558076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 949/1870] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:36.562209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 922/1871] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:36.574284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb9f0d-0ae1-4981-949e-6080e04c77c6 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:36.574668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb9f0d-0ae1-4981-949e-6080e04c77c6 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:36.575109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb9f0d-0ae1-4981-949e-6080e04c77c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:36.641287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-dffb9f0d-0ae1-4981-949e-6080e04c77c6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:36.642763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-dffb9f0d-0ae1-4981-949e-6080e04c77c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:36.692280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 950/1872] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:36.697291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887be4e9-aa7c-4179-89ab-732c11145e45 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:36.717382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887be4e9-aa7c-4179-89ab-732c11145e45 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:36.740335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89488522-6147-4d3e-a7a3-fa8d2e828a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:36.741064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488522-6147-4d3e-a7a3-fa8d2e828a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:36.745044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488522-6147-4d3e-a7a3-fa8d2e828a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:36.745994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488522-6147-4d3e-a7a3-fa8d2e828a7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:36.749870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488522-6147-4d3e-a7a3-fa8d2e828a7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:36.750524 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488522-6147-4d3e-a7a3-fa8d2e828a7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:36.754187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488522-6147-4d3e-a7a3-fa8d2e828a7c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:36.794013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 923/1873] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:36.796005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb9f0d-0ae1-4981-949e-6080e04c77c6 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:36.808932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb9f0d-0ae1-4981-949e-6080e04c77c6 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:36.825435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d665e-12e7-483f-b339-04f6122def33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:36.825435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d665e-12e7-483f-b339-04f6122def33 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:36.825435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d665e-12e7-483f-b339-04f6122def33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:36.825435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d665e-12e7-483f-b339-04f6122def33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:36.825856 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d665e-12e7-483f-b339-04f6122def33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:36.833693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d665e-12e7-483f-b339-04f6122def33 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:36.922955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89488522-6147-4d3e-a7a3-fa8d2e828a7c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:36.924600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 951/1874] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:36 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:36.943233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89488522-6147-4d3e-a7a3-fa8d2e828a7c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:37.166052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05acb0-0dd7-4656-a25f-afe7971a92d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:37.167156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05acb0-0dd7-4656-a25f-afe7971a92d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:37.167532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05acb0-0dd7-4656-a25f-afe7971a92d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:37.168693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05acb0-0dd7-4656-a25f-afe7971a92d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:37.169530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05acb0-0dd7-4656-a25f-afe7971a92d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:37.173127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05acb0-0dd7-4656-a25f-afe7971a92d0 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:37.236398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 924/1875] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:36 2022] POST /identity/v3/users => generated 289 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:30:37.237889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d665e-12e7-483f-b339-04f6122def33 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:37.258808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d665e-12e7-483f-b339-04f6122def33 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:37.272198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33f408-3353-47e8-8abb-3d8b00338687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:37.273738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33f408-3353-47e8-8abb-3d8b00338687 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:37.274977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33f408-3353-47e8-8abb-3d8b00338687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:37.275489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33f408-3353-47e8-8abb-3d8b00338687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:37.277023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33f408-3353-47e8-8abb-3d8b00338687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:37.279981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33f408-3353-47e8-8abb-3d8b00338687 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:37.304919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 952/1876] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:37.307177 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05acb0-0dd7-4656-a25f-afe7971a92d0 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:37.325968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05acb0-0dd7-4656-a25f-afe7971a92d0 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:37.333008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d47cf5-364d-4a78-ab86-2e9473ce225b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:37.334631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d47cf5-364d-4a78-ab86-2e9473ce225b admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:37.335113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d47cf5-364d-4a78-ab86-2e9473ce225b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:37.335580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d47cf5-364d-4a78-ab86-2e9473ce225b admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:37.336464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d47cf5-364d-4a78-ab86-2e9473ce225b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:37.339307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d47cf5-364d-4a78-ab86-2e9473ce225b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:37.361230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 925/1877] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:37.365392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33f408-3353-47e8-8abb-3d8b00338687 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:37.379259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33f408-3353-47e8-8abb-3d8b00338687 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:37.389125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c36c9-4735-4552-801b-fb9da23ebecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:37.390613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c36c9-4735-4552-801b-fb9da23ebecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:37.406626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c36c9-4735-4552-801b-fb9da23ebecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:37.407183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c36c9-4735-4552-801b-fb9da23ebecb admin admin] PATH_INFO: `/v3/projects/390089c1038e43b19e1fb852e4a449e1/users/d37913b7bfdd40f5a9ebecff036a177c/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:37.449859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c36c9-4735-4552-801b-fb9da23ebecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=390089c1038e43b19e1fb852e4a449e1, user_id=d37913b7bfdd40f5a9ebecff036a177c, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:37.452548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c36c9-4735-4552-801b-fb9da23ebecb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:37.485212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 953/1878] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:37 2022] PUT /identity/v3/projects/390089c1038e43b19e1fb852e4a449e1/users/d37913b7bfdd40f5a9ebecff036a177c/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:37.492347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c509ce-d7d2-4419-afe5-6f6ddc90783f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:37.492668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c509ce-d7d2-4419-afe5-6f6ddc90783f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:37.493223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c509ce-d7d2-4419-afe5-6f6ddc90783f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:37.527670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-67c509ce-d7d2-4419-afe5-6f6ddc90783f None None] MFA Rules not processed for user `d37913b7bfdd40f5a9ebecff036a177c`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:37.528970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-67c509ce-d7d2-4419-afe5-6f6ddc90783f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:37.535234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:37.645523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 926/1879] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:37.650596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c36c9-4735-4552-801b-fb9da23ebecb admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:37.754509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c36c9-4735-4552-801b-fb9da23ebecb admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:37.760292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee9ca-0b7a-42e9-81f6-56cd9de7c6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:37.761275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee9ca-0b7a-42e9-81f6-56cd9de7c6fa admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:37.761605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee9ca-0b7a-42e9-81f6-56cd9de7c6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:37.762106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee9ca-0b7a-42e9-81f6-56cd9de7c6fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:37.762822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee9ca-0b7a-42e9-81f6-56cd9de7c6fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:37.764761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee9ca-0b7a-42e9-81f6-56cd9de7c6fa admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:37.800600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 954/1880] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:37 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:37.805221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c509ce-d7d2-4419-afe5-6f6ddc90783f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:37.816690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c509ce-d7d2-4419-afe5-6f6ddc90783f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:37.893018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1c72f-3ed2-477a-85d8-fe6a26c5818b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:37.894111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1c72f-3ed2-477a-85d8-fe6a26c5818b admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:37.894639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1c72f-3ed2-477a-85d8-fe6a26c5818b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:37.895914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1c72f-3ed2-477a-85d8-fe6a26c5818b admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:37.896778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1c72f-3ed2-477a-85d8-fe6a26c5818b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:37.899053 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1c72f-3ed2-477a-85d8-fe6a26c5818b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:37.924018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 927/1881] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:37 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:37.929578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee9ca-0b7a-42e9-81f6-56cd9de7c6fa admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:37.942806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee9ca-0b7a-42e9-81f6-56cd9de7c6fa admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:37.947386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37fefc-8975-4f9a-95cc-524c57c01439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:37.948131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37fefc-8975-4f9a-95cc-524c57c01439 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:37.948634 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37fefc-8975-4f9a-95cc-524c57c01439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:37.949261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37fefc-8975-4f9a-95cc-524c57c01439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:37.950090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37fefc-8975-4f9a-95cc-524c57c01439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:37.951734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37fefc-8975-4f9a-95cc-524c57c01439 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:37.968098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 955/1882] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:37.971270 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1c72f-3ed2-477a-85d8-fe6a26c5818b admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:37.983696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1c72f-3ed2-477a-85d8-fe6a26c5818b admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:37.988968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e74c0f-257f-4739-aa09-718bdda98ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:37.989652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e74c0f-257f-4739-aa09-718bdda98ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:37.990041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e74c0f-257f-4739-aa09-718bdda98ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:37.990216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e74c0f-257f-4739-aa09-718bdda98ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:37.990938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e74c0f-257f-4739-aa09-718bdda98ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:37.992589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e74c0f-257f-4739-aa09-718bdda98ed5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:38.005036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 928/1883] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:38.011236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37fefc-8975-4f9a-95cc-524c57c01439 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:38.032211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37fefc-8975-4f9a-95cc-524c57c01439 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:38.039544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9714d5a-05e0-4c92-b9b1-c3b45751ab6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:38.041140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9714d5a-05e0-4c92-b9b1-c3b45751ab6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:38.041563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9714d5a-05e0-4c92-b9b1-c3b45751ab6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:38.041806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9714d5a-05e0-4c92-b9b1-c3b45751ab6b admin admin] PATH_INFO: `/v3/projects/2eb2250413fb4622b0e26f43e640a900/users/791e9a43864644db9c69ee9b0fa8be12/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:38.067546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9714d5a-05e0-4c92-b9b1-c3b45751ab6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb2250413fb4622b0e26f43e640a900, user_id=791e9a43864644db9c69ee9b0fa8be12, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:38.070712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9714d5a-05e0-4c92-b9b1-c3b45751ab6b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:38.092355 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 956/1884] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:38 2022] PUT /identity/v3/projects/2eb2250413fb4622b0e26f43e640a900/users/791e9a43864644db9c69ee9b0fa8be12/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:38.102873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f8c9-39d3-45a5-a92f-4ea3987eef3a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:38.103661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f8c9-39d3-45a5-a92f-4ea3987eef3a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:38.104026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f8c9-39d3-45a5-a92f-4ea3987eef3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:38.153924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-16e0f8c9-39d3-45a5-a92f-4ea3987eef3a None None] MFA Rules not processed for user `791e9a43864644db9c69ee9b0fa8be12`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:38.155160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-16e0f8c9-39d3-45a5-a92f-4ea3987eef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:38.269239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 929/1885] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:38.270264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9714d5a-05e0-4c92-b9b1-c3b45751ab6b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:38.319291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:38.435257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9714d5a-05e0-4c92-b9b1-c3b45751ab6b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:38.451578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9152f37-6176-4afc-978f-c1b2dade0870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:38.452452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152f37-6176-4afc-978f-c1b2dade0870 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:38.455714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152f37-6176-4afc-978f-c1b2dade0870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:38.456636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152f37-6176-4afc-978f-c1b2dade0870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:38.457530 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9152f37-6176-4afc-978f-c1b2dade0870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:38.461372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9152f37-6176-4afc-978f-c1b2dade0870 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:38.497712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f8c9-39d3-45a5-a92f-4ea3987eef3a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:38.499701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-16e0f8c9-39d3-45a5-a92f-4ea3987eef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:38.529351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 957/1886] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:38 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:38.530653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9152f37-6176-4afc-978f-c1b2dade0870 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:38.541857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f8c9-39d3-45a5-a92f-4ea3987eef3a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:38.567129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:38.664897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9152f37-6176-4afc-978f-c1b2dade0870 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:38.674389 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:38.675656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:38.676701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:38.677210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] PATH_INFO: `/v3/users/04b40efa87794c118513a9f72ee0972a` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:38.678987 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04b40efa87794c118513a9f72ee0972a)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:38.682032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ce54-3e79-49d5-a9d6-83194dbf9002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:38.682868 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ce54-3e79-49d5-a9d6-83194dbf9002 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:38.684334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ce54-3e79-49d5-a9d6-83194dbf9002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:38.685808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:38.688100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ce54-3e79-49d5-a9d6-83194dbf9002 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:38.690942 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ce54-3e79-49d5-a9d6-83194dbf9002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:38.714866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ce54-3e79-49d5-a9d6-83194dbf9002 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:38.772257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 958/1887] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:38 2022] POST /identity/v3/users => generated 296 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:38.773974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ce54-3e79-49d5-a9d6-83194dbf9002 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:38.787779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04b40efa87794c118513a9f72ee0972a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:38.789520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04b40efa87794c118513a9f72ee0972a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:38.793090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ce54-3e79-49d5-a9d6-83194dbf9002 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:38.801168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04b40efa87794c118513a9f72ee0972a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:38.802158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04b40efa87794c118513a9f72ee0972a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:38.806356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d70fd-b9c2-4520-8d0d-5ae640d99bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:38.807570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d70fd-b9c2-4520-8d0d-5ae640d99bef admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:38.807979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d70fd-b9c2-4520-8d0d-5ae640d99bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:38.808963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d70fd-b9c2-4520-8d0d-5ae640d99bef admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:38.810028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d70fd-b9c2-4520-8d0d-5ae640d99bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:38.814705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d70fd-b9c2-4520-8d0d-5ae640d99bef admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:38.828293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-25e0441a-9ffc-45b6-a1eb-fc5921dd9692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04b40efa87794c118513a9f72ee0972a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:38.843023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 959/1888] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:38.846001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d70fd-b9c2-4520-8d0d-5ae640d99bef admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:38.854804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-eb7d70fd-b9c2-4520-8d0d-5ae640d99bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:38.884519 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 930/1889] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:38 2022] DELETE /identity/v3/users/04b40efa87794c118513a9f72ee0972a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:38.887042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d70fd-b9c2-4520-8d0d-5ae640d99bef admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:38.887176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156aa32-03e5-4545-a9e6-7943526a7783 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:38.887431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156aa32-03e5-4545-a9e6-7943526a7783 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:38.887661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156aa32-03e5-4545-a9e6-7943526a7783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:38.914473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-5156aa32-03e5-4545-a9e6-7943526a7783 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:38.915849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5156aa32-03e5-4545-a9e6-7943526a7783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:39.040515 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462a238-3c72-4525-808e-08b5206e6a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:39.041477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462a238-3c72-4525-808e-08b5206e6a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:39.041761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462a238-3c72-4525-808e-08b5206e6a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:39.043896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462a238-3c72-4525-808e-08b5206e6a5c admin admin] PATH_INFO: `/v3/projects/43148bd16def484ba9aedcc95febec7c/users/1e6e9e17a8e442fa91e51856537c0e04/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:39.051665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 931/1890] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:39.083029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462a238-3c72-4525-808e-08b5206e6a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43148bd16def484ba9aedcc95febec7c, user_id=1e6e9e17a8e442fa91e51856537c0e04, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:39.086384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462a238-3c72-4525-808e-08b5206e6a5c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:39.096654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156aa32-03e5-4545-a9e6-7943526a7783 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:39.102040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5156aa32-03e5-4545-a9e6-7943526a7783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:39.135630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 960/1891] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:38 2022] PUT /identity/v3/projects/43148bd16def484ba9aedcc95febec7c/users/1e6e9e17a8e442fa91e51856537c0e04/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:39.138549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462a238-3c72-4525-808e-08b5206e6a5c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:39.173219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:39.288752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:39.340369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156aa32-03e5-4545-a9e6-7943526a7783 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:39.353398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462a238-3c72-4525-808e-08b5206e6a5c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:39.363444 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddcad9-aae0-4aa7-a4a6-9c544dbac059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:39.364224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddcad9-aae0-4aa7-a4a6-9c544dbac059 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:39.364511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddcad9-aae0-4aa7-a4a6-9c544dbac059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:39.366796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddcad9-aae0-4aa7-a4a6-9c544dbac059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:39.367285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ddcad9-aae0-4aa7-a4a6-9c544dbac059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:39.368967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ddcad9-aae0-4aa7-a4a6-9c544dbac059 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:39.393559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 961/1892] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:39.401216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49e6db-d3fa-4582-b87a-ec5fe0ba1644 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:39.401507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49e6db-d3fa-4582-b87a-ec5fe0ba1644 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:39.401790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49e6db-d3fa-4582-b87a-ec5fe0ba1644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:39.472510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3453b-8b6b-4fc5-8fb1-9140b4564869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:39.472707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1b49e6db-d3fa-4582-b87a-ec5fe0ba1644 None None] MFA Rules not processed for user `767b5629a2ce472cb57ebc2421597338`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:39.474171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1b49e6db-d3fa-4582-b87a-ec5fe0ba1644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:39.475779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3453b-8b6b-4fc5-8fb1-9140b4564869 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:39.477448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3453b-8b6b-4fc5-8fb1-9140b4564869 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:39.478411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3453b-8b6b-4fc5-8fb1-9140b4564869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:39.503922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3453b-8b6b-4fc5-8fb1-9140b4564869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:39.506939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3453b-8b6b-4fc5-8fb1-9140b4564869 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:39.539560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 932/1893] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:39.540484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3453b-8b6b-4fc5-8fb1-9140b4564869 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:39.567119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3453b-8b6b-4fc5-8fb1-9140b4564869 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:39.582941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22745ea6-29df-4563-8ebd-eb72c1bad85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:39.584828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22745ea6-29df-4563-8ebd-eb72c1bad85b admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:39.584828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22745ea6-29df-4563-8ebd-eb72c1bad85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:39.584828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22745ea6-29df-4563-8ebd-eb72c1bad85b admin admin] PATH_INFO: `/v3/projects/43148bd16def484ba9aedcc95febec7c/users/1e6e9e17a8e442fa91e51856537c0e04/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:39.593291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 962/1894] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:39.596465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22745ea6-29df-4563-8ebd-eb72c1bad85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=43148bd16def484ba9aedcc95febec7c, user_id=1e6e9e17a8e442fa91e51856537c0e04, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:39.596564 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49e6db-d3fa-4582-b87a-ec5fe0ba1644 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:39.599540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1b49e6db-d3fa-4582-b87a-ec5fe0ba1644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:39.600980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22745ea6-29df-4563-8ebd-eb72c1bad85b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:39.630143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 933/1895] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:39 2022] PUT /identity/v3/projects/43148bd16def484ba9aedcc95febec7c/users/1e6e9e17a8e442fa91e51856537c0e04/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:39.631839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49e6db-d3fa-4582-b87a-ec5fe0ba1644 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:39.643117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921f021-4266-4597-9382-03835a55225f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:39.643491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921f021-4266-4597-9382-03835a55225f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:39.643850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921f021-4266-4597-9382-03835a55225f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:39.691649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-f921f021-4266-4597-9382-03835a55225f None None] MFA Rules not processed for user `1e6e9e17a8e442fa91e51856537c0e04`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:39.695976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f921f021-4266-4597-9382-03835a55225f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:39.797719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf3d7b-cf4f-4742-8fb8-f46f0987ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:39.798398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf3d7b-cf4f-4742-8fb8-f46f0987ed2f admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:39.798682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf3d7b-cf4f-4742-8fb8-f46f0987ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:39.799558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf3d7b-cf4f-4742-8fb8-f46f0987ed2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:39.800281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf3d7b-cf4f-4742-8fb8-f46f0987ed2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:39.818971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf3d7b-cf4f-4742-8fb8-f46f0987ed2f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:39.878891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 963/1896] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:39 2022] POST /identity/v3/projects => generated 371 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:39.885317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf3d7b-cf4f-4742-8fb8-f46f0987ed2f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:39.942788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 934/1897] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:39.954272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921f021-4266-4597-9382-03835a55225f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:39.957298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f921f021-4266-4597-9382-03835a55225f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:39.974821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:40.111360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf3d7b-cf4f-4742-8fb8-f46f0987ed2f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:40.131811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25459394-b1c0-4ff3-82ba-f9ce2a3666a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:40.133279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459394-b1c0-4ff3-82ba-f9ce2a3666a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:40.133493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459394-b1c0-4ff3-82ba-f9ce2a3666a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:40.133771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459394-b1c0-4ff3-82ba-f9ce2a3666a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:40.134517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25459394-b1c0-4ff3-82ba-f9ce2a3666a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:40.137123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25459394-b1c0-4ff3-82ba-f9ce2a3666a3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:40.156411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921f021-4266-4597-9382-03835a55225f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:40.183044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 964/1898] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:39 2022] POST /identity/v3/users => generated 292 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:40.186575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25459394-b1c0-4ff3-82ba-f9ce2a3666a3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:40.207117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:40.214454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25459394-b1c0-4ff3-82ba-f9ce2a3666a3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:40.227257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff3b31-f5ac-4ba1-ace8-37cd12872801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:40.228073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff3b31-f5ac-4ba1-ace8-37cd12872801 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:40.228548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff3b31-f5ac-4ba1-ace8-37cd12872801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:40.228835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff3b31-f5ac-4ba1-ace8-37cd12872801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:40.229607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff3b31-f5ac-4ba1-ace8-37cd12872801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:40.232672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff3b31-f5ac-4ba1-ace8-37cd12872801 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:40.265971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12dc8-6ab0-4404-94c5-179cc323b7db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:40.266916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12dc8-6ab0-4404-94c5-179cc323b7db service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:40.267456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12dc8-6ab0-4404-94c5-179cc323b7db service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:40.267456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12dc8-6ab0-4404-94c5-179cc323b7db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:40.284970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 965/1899] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:40.292113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff3b31-f5ac-4ba1-ace8-37cd12872801 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:40.334450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff3b31-f5ac-4ba1-ace8-37cd12872801 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:40.359155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3855e-85ed-4fbc-bb2a-fbf4bf212e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:40.360661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3855e-85ed-4fbc-bb2a-fbf4bf212e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:40.363624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3855e-85ed-4fbc-bb2a-fbf4bf212e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:40.364105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3855e-85ed-4fbc-bb2a-fbf4bf212e87 admin admin] PATH_INFO: `/v3/projects/a3217b6fd0f24e23958838338a3e7ea4/users/977937e98ffd4308a39bc605b7f7fe94/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:40.421282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3855e-85ed-4fbc-bb2a-fbf4bf212e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3217b6fd0f24e23958838338a3e7ea4, user_id=977937e98ffd4308a39bc605b7f7fe94, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:40.424132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3855e-85ed-4fbc-bb2a-fbf4bf212e87 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:40.443224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e12dc8-6ab0-4404-94c5-179cc323b7db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:40.444700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e12dc8-6ab0-4404-94c5-179cc323b7db service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:40.446299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 966/1900] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:40 2022] PUT /identity/v3/projects/a3217b6fd0f24e23958838338a3e7ea4/users/977937e98ffd4308a39bc605b7f7fe94/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:40.449986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3855e-85ed-4fbc-bb2a-fbf4bf212e87 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:40.493925 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 935/1901] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:40.549243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3855e-85ed-4fbc-bb2a-fbf4bf212e87 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:40.555963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea1aea-c1e4-4ee0-bd03-9374848c27f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:40.558029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea1aea-c1e4-4ee0-bd03-9374848c27f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:40.558424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea1aea-c1e4-4ee0-bd03-9374848c27f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:40.558720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea1aea-c1e4-4ee0-bd03-9374848c27f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:40.559726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea1aea-c1e4-4ee0-bd03-9374848c27f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:40.563217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea1aea-c1e4-4ee0-bd03-9374848c27f5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:40.578886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12dc8-6ab0-4404-94c5-179cc323b7db service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:40.580477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-93e12dc8-6ab0-4404-94c5-179cc323b7db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:40.594785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 967/1902] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:40.594785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea1aea-c1e4-4ee0-bd03-9374848c27f5 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:40.608710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12dc8-6ab0-4404-94c5-179cc323b7db service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:40.623971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea1aea-c1e4-4ee0-bd03-9374848c27f5 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:40.639176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3080d6-593f-4f14-80ee-5e64c2935287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:40.640404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3080d6-593f-4f14-80ee-5e64c2935287 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:40.640737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3080d6-593f-4f14-80ee-5e64c2935287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:40.641788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3080d6-593f-4f14-80ee-5e64c2935287 admin admin] PATH_INFO: `/v3/projects/a3217b6fd0f24e23958838338a3e7ea4/users/977937e98ffd4308a39bc605b7f7fe94/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:40.648250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3080d6-593f-4f14-80ee-5e64c2935287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3217b6fd0f24e23958838338a3e7ea4, user_id=977937e98ffd4308a39bc605b7f7fe94, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:40.651174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3080d6-593f-4f14-80ee-5e64c2935287 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:40.680374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 968/1903] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:40 2022] PUT /identity/v3/projects/a3217b6fd0f24e23958838338a3e7ea4/users/977937e98ffd4308a39bc605b7f7fe94/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:40.731927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a76b66-3bb0-406b-ac66-ac1c8c1ed83a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:40.732589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a76b66-3bb0-406b-ac66-ac1c8c1ed83a service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:40.732888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a76b66-3bb0-406b-ac66-ac1c8c1ed83a service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:40.733350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a76b66-3bb0-406b-ac66-ac1c8c1ed83a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:40.742495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-58a76b66-3bb0-406b-ac66-ac1c8c1ed83a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:40.856091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a76b66-3bb0-406b-ac66-ac1c8c1ed83a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:40.859199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a76b66-3bb0-406b-ac66-ac1c8c1ed83a service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:40.883261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 936/1904] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:41.103243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3080d6-593f-4f14-80ee-5e64c2935287 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:41.105574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2d3080d6-593f-4f14-80ee-5e64c2935287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:41.273392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3080d6-593f-4f14-80ee-5e64c2935287 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:41.287691 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:41.288610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:41.288910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:41.289183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] PATH_INFO: `/v3/projects/80efdb219409498f9ad4e35f08790314` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:41.293942 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80efdb219409498f9ad4e35f08790314)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:41.295889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:41.297662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:41.402910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80efdb219409498f9ad4e35f08790314', 'request_id': 'req-85573577-92f3-42b2-8f70-c43722bb6749'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.414315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80efdb219409498f9ad4e35f08790314', 'request_id': 'req-85573577-92f3-42b2-8f70-c43722bb6749'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.447579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80efdb219409498f9ad4e35f08790314', 'request_id': 'req-85573577-92f3-42b2-8f70-c43722bb6749'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.473311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] The token cache is being invalidate because project 80efdb219409498f9ad4e35f08790314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:41.473786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.475015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-85573577-92f3-42b2-8f70-c43722bb6749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.483496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 969/1905] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:41 2022] DELETE /identity/v3/projects/80efdb219409498f9ad4e35f08790314 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:41.484831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a76b66-3bb0-406b-ac66-ac1c8c1ed83a service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:41.497616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a76b66-3bb0-406b-ac66-ac1c8c1ed83a service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:41.595420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:41.784705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:41.787769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:41.790610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:41.791256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] PATH_INFO: `/v3/users/4c3d401a5b4d4aeb99573863be53e385` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:41.794229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3d401a5b4d4aeb99573863be53e385)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:41.802199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:41.926903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3d401a5b4d4aeb99573863be53e385'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.927725 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3d401a5b4d4aeb99573863be53e385'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.938748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3d401a5b4d4aeb99573863be53e385'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.939765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3d401a5b4d4aeb99573863be53e385'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.955636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3d401a5b4d4aeb99573863be53e385'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:41.983195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 937/1906] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:41 2022] DELETE /identity/v3/users/4c3d401a5b4d4aeb99573863be53e385 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:42.317941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd09b3-fede-4003-92e2-10f5256edd71 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:42.318498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd09b3-fede-4003-92e2-10f5256edd71 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:42.318575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd09b3-fede-4003-92e2-10f5256edd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:42.379193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-10dd09b3-fede-4003-92e2-10f5256edd71 None None] MFA Rules not processed for user `977937e98ffd4308a39bc605b7f7fe94`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:42.380472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-10dd09b3-fede-4003-92e2-10f5256edd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:42.631867 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 970/1907] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:42.645029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:42.648048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:42.761810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854fb65-1ca9-40fd-8838-727214b5daa6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:42.767067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9acf6-28f3-40d8-a873-a671758cd9f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:42.768063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9acf6-28f3-40d8-a873-a671758cd9f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:42.768646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9acf6-28f3-40d8-a873-a671758cd9f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:42.770498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9acf6-28f3-40d8-a873-a671758cd9f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:42.781198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c1c9acf6-28f3-40d8-a873-a671758cd9f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:42.899653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9acf6-28f3-40d8-a873-a671758cd9f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:42.901336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9acf6-28f3-40d8-a873-a671758cd9f3 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:42.909915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:42.927401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 938/1908] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:44.063127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd09b3-fede-4003-92e2-10f5256edd71 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.064744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-10dd09b3-fede-4003-92e2-10f5256edd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:44.154784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd09b3-fede-4003-92e2-10f5256edd71 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:44.161518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:44.162661 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.163046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.163945 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] PATH_INFO: `/v3/projects/1f57aca0025b4ee19fd4f70a5cd77dc3` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.165304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f57aca0025b4ee19fd4f70a5cd77dc3)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:44.167304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:44.189653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:44.227044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f57aca0025b4ee19fd4f70a5cd77dc3', 'request_id': 'req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:44.234451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f57aca0025b4ee19fd4f70a5cd77dc3', 'request_id': 'req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:44.240437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f57aca0025b4ee19fd4f70a5cd77dc3', 'request_id': 'req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:44.257487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] The token cache is being invalidate because project 1f57aca0025b4ee19fd4f70a5cd77dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:44.257790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:44.258651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d946bd7d-8eff-4753-aa1b-5a3be3dc2bbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:44.260815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 971/1909] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:44 2022] DELETE /identity/v3/projects/1f57aca0025b4ee19fd4f70a5cd77dc3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:44.266764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7fcaf-438a-4639-be4e-dc95d175d61a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.267663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7fcaf-438a-4639-be4e-dc95d175d61a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.268021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7fcaf-438a-4639-be4e-dc95d175d61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.289099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a7b7fcaf-438a-4639-be4e-dc95d175d61a None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:44.290426 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a7b7fcaf-438a-4639-be4e-dc95d175d61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:44.359530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 939/1910] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:44.367474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f013f-1f84-41c8-b1b8-f7e4be1b5a11 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.367772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f013f-1f84-41c8-b1b8-f7e4be1b5a11 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.368055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f013f-1f84-41c8-b1b8-f7e4be1b5a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.388828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-237f013f-1f84-41c8-b1b8-f7e4be1b5a11 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:44.390084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-237f013f-1f84-41c8-b1b8-f7e4be1b5a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:44.398831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 972/1911] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:44.403186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7fcaf-438a-4639-be4e-dc95d175d61a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.413332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7fcaf-438a-4639-be4e-dc95d175d61a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:44.419518 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af379e6c-29f7-458a-b742-555c187fdec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:44.420215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af379e6c-29f7-458a-b742-555c187fdec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.420622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af379e6c-29f7-458a-b742-555c187fdec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.421087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af379e6c-29f7-458a-b742-555c187fdec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.422075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af379e6c-29f7-458a-b742-555c187fdec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:44.422405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af379e6c-29f7-458a-b742-555c187fdec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:44.424173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af379e6c-29f7-458a-b742-555c187fdec3 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:44.437004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 940/1912] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:44.441398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f013f-1f84-41c8-b1b8-f7e4be1b5a11 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.450958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f013f-1f84-41c8-b1b8-f7e4be1b5a11 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:44.455596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cb9db-f472-4225-9db5-e46716d3ee0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:44.456254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cb9db-f472-4225-9db5-e46716d3ee0b admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.456532 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cb9db-f472-4225-9db5-e46716d3ee0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.456807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cb9db-f472-4225-9db5-e46716d3ee0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.457547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cb9db-f472-4225-9db5-e46716d3ee0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:44.459452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cb9db-f472-4225-9db5-e46716d3ee0b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:44.489314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 973/1913] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:44 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:30:44.490490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af379e6c-29f7-458a-b742-555c187fdec3 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.501197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af379e6c-29f7-458a-b742-555c187fdec3 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:44.567231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d493bc1-8318-4680-affd-b273123f556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:44.568067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d493bc1-8318-4680-affd-b273123f556d admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.568419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d493bc1-8318-4680-affd-b273123f556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.568765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d493bc1-8318-4680-affd-b273123f556d admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.569552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d493bc1-8318-4680-affd-b273123f556d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:44.571704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d493bc1-8318-4680-affd-b273123f556d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:44.601394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cb9db-f472-4225-9db5-e46716d3ee0b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.606092 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 941/1914] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:44 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:44.606284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5f7cb9db-f472-4225-9db5-e46716d3ee0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:44.607612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d493bc1-8318-4680-affd-b273123f556d admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.618692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d493bc1-8318-4680-affd-b273123f556d admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:44.626407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1e213-c00b-4038-93e9-77bb50d5f30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:44.628624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1e213-c00b-4038-93e9-77bb50d5f30e admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.628972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1e213-c00b-4038-93e9-77bb50d5f30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.629263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1e213-c00b-4038-93e9-77bb50d5f30e admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.630013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1e213-c00b-4038-93e9-77bb50d5f30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:44.633861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1e213-c00b-4038-93e9-77bb50d5f30e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:44.650178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 942/1915] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:44.653241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1e213-c00b-4038-93e9-77bb50d5f30e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.665027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1e213-c00b-4038-93e9-77bb50d5f30e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:44.671678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d80348-37d3-47bb-bc90-741f8b87752b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:44.672389 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d80348-37d3-47bb-bc90-741f8b87752b admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.673232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d80348-37d3-47bb-bc90-741f8b87752b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.673513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d80348-37d3-47bb-bc90-741f8b87752b admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.674420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d80348-37d3-47bb-bc90-741f8b87752b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:44.676475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d80348-37d3-47bb-bc90-741f8b87752b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:44.678822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cb9db-f472-4225-9db5-e46716d3ee0b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:44.682980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fe9c9-13df-42f4-b043-91bd3ac1d7bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:44.684007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fe9c9-13df-42f4-b043-91bd3ac1d7bb service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.684389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fe9c9-13df-42f4-b043-91bd3ac1d7bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.684769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fe9c9-13df-42f4-b043-91bd3ac1d7bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.687036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1a7fe9c9-13df-42f4-b043-91bd3ac1d7bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:44.691530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 943/1916] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:44.696735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d80348-37d3-47bb-bc90-741f8b87752b admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.710650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d80348-37d3-47bb-bc90-741f8b87752b admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:44.718183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92c806-0a6e-4785-acee-a8675a3ebdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:44.719173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92c806-0a6e-4785-acee-a8675a3ebdc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.719455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92c806-0a6e-4785-acee-a8675a3ebdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.720005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92c806-0a6e-4785-acee-a8675a3ebdc2 admin admin] PATH_INFO: `/v3/projects/d4175fb2ca824ebeb2dae1ec9396252c/users/24e9753bc61849a6b3c0f4ba29b18efa/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.740473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92c806-0a6e-4785-acee-a8675a3ebdc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4175fb2ca824ebeb2dae1ec9396252c, user_id=24e9753bc61849a6b3c0f4ba29b18efa, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:44.743790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92c806-0a6e-4785-acee-a8675a3ebdc2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:44.762572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 944/1917] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:44 2022] PUT /identity/v3/projects/d4175fb2ca824ebeb2dae1ec9396252c/users/24e9753bc61849a6b3c0f4ba29b18efa/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:44.773201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96585d0d-b4ef-44bc-916d-af21c734f50a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:44.773491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96585d0d-b4ef-44bc-916d-af21c734f50a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:44.773767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96585d0d-b4ef-44bc-916d-af21c734f50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:44.786566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fe9c9-13df-42f4-b043-91bd3ac1d7bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:44.788231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fe9c9-13df-42f4-b043-91bd3ac1d7bb service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:44.809686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 974/1918] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:44.809899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-96585d0d-b4ef-44bc-916d-af21c734f50a None None] MFA Rules not processed for user `24e9753bc61849a6b3c0f4ba29b18efa`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:44.811117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-96585d0d-b4ef-44bc-916d-af21c734f50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:44.834740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:44.981034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 945/1919] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:44.985145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fe9c9-13df-42f4-b043-91bd3ac1d7bb service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.989991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96585d0d-b4ef-44bc-916d-af21c734f50a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:44.991486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-96585d0d-b4ef-44bc-916d-af21c734f50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:45.000184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fe9c9-13df-42f4-b043-91bd3ac1d7bb service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:45.043142 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:45.125752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96585d0d-b4ef-44bc-916d-af21c734f50a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:45.130822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:45.131713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.132086 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.132260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] PATH_INFO: `/v3/users/d37913b7bfdd40f5a9ebecff036a177c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.133934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37913b7bfdd40f5a9ebecff036a177c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:45.137119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:45.205629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64808424-ceb5-4d30-a5bd-1d114cecda47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:45.208133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64808424-ceb5-4d30-a5bd-1d114cecda47 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.211114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64808424-ceb5-4d30-a5bd-1d114cecda47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.211539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64808424-ceb5-4d30-a5bd-1d114cecda47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.212395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64808424-ceb5-4d30-a5bd-1d114cecda47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:45.219125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64808424-ceb5-4d30-a5bd-1d114cecda47 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:45.230956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37913b7bfdd40f5a9ebecff036a177c'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:45.231703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37913b7bfdd40f5a9ebecff036a177c'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:45.240443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37913b7bfdd40f5a9ebecff036a177c'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:45.241743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37913b7bfdd40f5a9ebecff036a177c'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:45.274037 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-58978121-dc9e-4fd7-b7d7-7b13f06896fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37913b7bfdd40f5a9ebecff036a177c'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:45.299767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 946/1920] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:44 2022] DELETE /identity/v3/users/d37913b7bfdd40f5a9ebecff036a177c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:45.305893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 975/1921] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:44 2022] POST /identity/v3/projects => generated 381 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:45.307732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64808424-ceb5-4d30-a5bd-1d114cecda47 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:45.307863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9c46-bfc1-48f4-a7f2-237af47ce3cc None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.308101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9c46-bfc1-48f4-a7f2-237af47ce3cc None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.308383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9c46-bfc1-48f4-a7f2-237af47ce3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.310286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-64808424-ceb5-4d30-a5bd-1d114cecda47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:45.343247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ae4d9c46-bfc1-48f4-a7f2-237af47ce3cc None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:45.343935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ae4d9c46-bfc1-48f4-a7f2-237af47ce3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:45.457928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:45.478219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64808424-ceb5-4d30-a5bd-1d114cecda47 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:45.485156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb152d-808a-44c1-ad88-ef157ebb46b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:45.485852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb152d-808a-44c1-ad88-ef157ebb46b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.486138 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb152d-808a-44c1-ad88-ef157ebb46b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.486416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb152d-808a-44c1-ad88-ef157ebb46b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.487201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb152d-808a-44c1-ad88-ef157ebb46b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:45.489189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb152d-808a-44c1-ad88-ef157ebb46b7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:45.493323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 947/1922] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:45.504399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d9c46-bfc1-48f4-a7f2-237af47ce3cc None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:45.506222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ae4d9c46-bfc1-48f4-a7f2-237af47ce3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:45.513266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 976/1923] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:45 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:45.517578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb152d-808a-44c1-ad88-ef157ebb46b7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:45.530430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb152d-808a-44c1-ad88-ef157ebb46b7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:45.537576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a23bd8-55d1-42f6-acc3-4f27f176dd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:45.539167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a23bd8-55d1-42f6-acc3-4f27f176dd0c admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.539462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a23bd8-55d1-42f6-acc3-4f27f176dd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.539746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a23bd8-55d1-42f6-acc3-4f27f176dd0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.540908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a23bd8-55d1-42f6-acc3-4f27f176dd0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:45.543332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a23bd8-55d1-42f6-acc3-4f27f176dd0c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:45.555046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 977/1924] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:45.559619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a23bd8-55d1-42f6-acc3-4f27f176dd0c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:45.572360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a23bd8-55d1-42f6-acc3-4f27f176dd0c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:45.577831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c0435-4174-4159-8394-2d123dabce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:45.579034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d9c46-bfc1-48f4-a7f2-237af47ce3cc None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:45.579205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c0435-4174-4159-8394-2d123dabce3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.579563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c0435-4174-4159-8394-2d123dabce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.579941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c0435-4174-4159-8394-2d123dabce3a admin admin] PATH_INFO: `/v3/projects/8689ec9e1f43415699e0ed9be7dfa6ea/users/184efe9030864141bbff2d4a3d056266/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.583856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88759ad9-871a-4b53-836f-bb59f6c781ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:45.585381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88759ad9-871a-4b53-836f-bb59f6c781ac service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.585700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88759ad9-871a-4b53-836f-bb59f6c781ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.585980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88759ad9-871a-4b53-836f-bb59f6c781ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.598614 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c0435-4174-4159-8394-2d123dabce3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8689ec9e1f43415699e0ed9be7dfa6ea, user_id=184efe9030864141bbff2d4a3d056266, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:45.599548 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88759ad9-871a-4b53-836f-bb59f6c781ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:45.601330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88759ad9-871a-4b53-836f-bb59f6c781ac service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:45.602071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c0435-4174-4159-8394-2d123dabce3a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:45.613850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 948/1925] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:45.623770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 978/1926] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:45 2022] PUT /identity/v3/projects/8689ec9e1f43415699e0ed9be7dfa6ea/users/184efe9030864141bbff2d4a3d056266/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:45.627775 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88759ad9-871a-4b53-836f-bb59f6c781ac service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:45.639388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88759ad9-871a-4b53-836f-bb59f6c781ac service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:45.711967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b4f26-ae76-4151-91bb-1359535db5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:45.713113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b4f26-ae76-4151-91bb-1359535db5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.713562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b4f26-ae76-4151-91bb-1359535db5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.714001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b4f26-ae76-4151-91bb-1359535db5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.715351 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b4f26-ae76-4151-91bb-1359535db5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:45.717484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b4f26-ae76-4151-91bb-1359535db5a1 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:45.733231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 949/1927] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:45.734697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c0435-4174-4159-8394-2d123dabce3a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:45.746349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c0435-4174-4159-8394-2d123dabce3a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:45.752371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e7e73-c91f-4783-8221-f1fbf7090e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:45.753475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e7e73-c91f-4783-8221-f1fbf7090e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.753819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e7e73-c91f-4783-8221-f1fbf7090e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.754392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e7e73-c91f-4783-8221-f1fbf7090e54 admin admin] PATH_INFO: `/v3/projects/8689ec9e1f43415699e0ed9be7dfa6ea/users/184efe9030864141bbff2d4a3d056266/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.756809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e7e73-c91f-4783-8221-f1fbf7090e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8689ec9e1f43415699e0ed9be7dfa6ea, user_id=184efe9030864141bbff2d4a3d056266, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:45.759707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e7e73-c91f-4783-8221-f1fbf7090e54 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:45.775114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 979/1928] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:45 2022] PUT /identity/v3/projects/8689ec9e1f43415699e0ed9be7dfa6ea/users/184efe9030864141bbff2d4a3d056266/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:45.787345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08e035-9a32-4b45-9ae7-8d58350179d9 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:45.787631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08e035-9a32-4b45-9ae7-8d58350179d9 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:45.787924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08e035-9a32-4b45-9ae7-8d58350179d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:45.830275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ec08e035-9a32-4b45-9ae7-8d58350179d9 None None] MFA Rules not processed for user `184efe9030864141bbff2d4a3d056266`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:45.831694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ec08e035-9a32-4b45-9ae7-8d58350179d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:45.946424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e7e73-c91f-4783-8221-f1fbf7090e54 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:45.949171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3c8e7e73-c91f-4783-8221-f1fbf7090e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:45.968955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 950/1929] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:45.971421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08e035-9a32-4b45-9ae7-8d58350179d9 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:45.977558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ec08e035-9a32-4b45-9ae7-8d58350179d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:45.985071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:46.025860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:46.043836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e7e73-c91f-4783-8221-f1fbf7090e54 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:46.050983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:46.051489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08e035-9a32-4b45-9ae7-8d58350179d9 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:46.051994 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:46.052281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:46.052550 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] PATH_INFO: `/v3/projects/390089c1038e43b19e1fb852e4a449e1` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:46.053846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=390089c1038e43b19e1fb852e4a449e1)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:46.055784 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:46.055890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e38452-d37d-4d1d-aac9-93a321605108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:46.056459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e38452-d37d-4d1d-aac9-93a321605108 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:46.056743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e38452-d37d-4d1d-aac9-93a321605108 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:46.057058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e38452-d37d-4d1d-aac9-93a321605108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:46.068828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e38452-d37d-4d1d-aac9-93a321605108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:46.071210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e38452-d37d-4d1d-aac9-93a321605108 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:46.084708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 951/1930] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:46.108123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '390089c1038e43b19e1fb852e4a449e1', 'request_id': 'req-235aaa70-b563-46c6-aede-7b9b746af5d0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.114752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '390089c1038e43b19e1fb852e4a449e1', 'request_id': 'req-235aaa70-b563-46c6-aede-7b9b746af5d0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.123149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '390089c1038e43b19e1fb852e4a449e1', 'request_id': 'req-235aaa70-b563-46c6-aede-7b9b746af5d0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.139232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] The token cache is being invalidate because project 390089c1038e43b19e1fb852e4a449e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:46.139535 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.140187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.142034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 980/1931] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:45 2022] DELETE /identity/v3/projects/390089c1038e43b19e1fb852e4a449e1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:46.145936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e38452-d37d-4d1d-aac9-93a321605108 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:46.157650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e38452-d37d-4d1d-aac9-93a321605108 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:46.181968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:46.183993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:46.225715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:46.226514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:46.226809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:46.227078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] PATH_INFO: `/v3/users/791e9a43864644db9c69ee9b0fa8be12` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:46.228292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=791e9a43864644db9c69ee9b0fa8be12)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:46.230530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:46.242076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235aaa70-b563-46c6-aede-7b9b746af5d0 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:46.246533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce068e-6161-4846-8639-a093b2c59c7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:46.247202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce068e-6161-4846-8639-a093b2c59c7a service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:46.247487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce068e-6161-4846-8639-a093b2c59c7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:46.247763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce068e-6161-4846-8639-a093b2c59c7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:46.250000 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f1ce068e-6161-4846-8639-a093b2c59c7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:46.294242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791e9a43864644db9c69ee9b0fa8be12'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.295111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791e9a43864644db9c69ee9b0fa8be12'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.303625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791e9a43864644db9c69ee9b0fa8be12'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.305585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791e9a43864644db9c69ee9b0fa8be12'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.319905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791e9a43864644db9c69ee9b0fa8be12'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.327090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce068e-6161-4846-8639-a093b2c59c7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:46.328618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce068e-6161-4846-8639-a093b2c59c7a service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:46.337655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 952/1932] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:46 2022] DELETE /identity/v3/users/791e9a43864644db9c69ee9b0fa8be12 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:46.338790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 981/1933] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:46.769836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:46.771567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:46.815722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:46.868274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4353b-f169-4566-96d8-cd79a2cc7f8a admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:46.873601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:46.874714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:46.875155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:46.875532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] PATH_INFO: `/v3/users/24e9753bc61849a6b3c0f4ba29b18efa` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:46.877072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e9753bc61849a6b3c0f4ba29b18efa)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:46.879287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:46.941874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e9753bc61849a6b3c0f4ba29b18efa'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.942713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e9753bc61849a6b3c0f4ba29b18efa'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.953377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e9753bc61849a6b3c0f4ba29b18efa'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.954129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e9753bc61849a6b3c0f4ba29b18efa'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.977757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e9753bc61849a6b3c0f4ba29b18efa'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:46.996279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 953/1934] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:46 2022] DELETE /identity/v3/users/24e9753bc61849a6b3c0f4ba29b18efa => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:47.001105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29c3a2-22bd-465b-aeb7-556861d3b80e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:47.001393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29c3a2-22bd-465b-aeb7-556861d3b80e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:47.001690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29c3a2-22bd-465b-aeb7-556861d3b80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:47.024310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-8e29c3a2-22bd-465b-aeb7-556861d3b80e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:47.026166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8e29c3a2-22bd-465b-aeb7-556861d3b80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:47.070739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:47.115986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:47.117489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:47.127327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 982/1935] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:47.139478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29c3a2-22bd-465b-aeb7-556861d3b80e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:47.141588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8e29c3a2-22bd-465b-aeb7-556861d3b80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:47.220273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97144b84-89d2-4772-aab2-bd9371efcf1e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:47.225900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:47.227818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:47.228117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:47.228393 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] PATH_INFO: `/v3/projects/2eb2250413fb4622b0e26f43e640a900` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:47.229638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb2250413fb4622b0e26f43e640a900)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:47.232438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:47.243999 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29c3a2-22bd-465b-aeb7-556861d3b80e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:47.249248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c1c2c-b0fe-43a1-81d8-693bfe6a9e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:47.249930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c1c2c-b0fe-43a1-81d8-693bfe6a9e5c service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:47.250288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c1c2c-b0fe-43a1-81d8-693bfe6a9e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:47.250639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c1c2c-b0fe-43a1-81d8-693bfe6a9e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:47.263824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c1c2c-b0fe-43a1-81d8-693bfe6a9e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:47.265140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c1c2c-b0fe-43a1-81d8-693bfe6a9e5c service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:47.279870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 983/1936] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:47.288684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb2250413fb4622b0e26f43e640a900', 'request_id': 'req-adda1db0-6e75-4f21-afd6-a86d5ec1b260'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.289986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:47.297245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb2250413fb4622b0e26f43e640a900', 'request_id': 'req-adda1db0-6e75-4f21-afd6-a86d5ec1b260'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.309475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb2250413fb4622b0e26f43e640a900', 'request_id': 'req-adda1db0-6e75-4f21-afd6-a86d5ec1b260'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.341382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] The token cache is being invalidate because project 2eb2250413fb4622b0e26f43e640a900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:47.341674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.342355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.344412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 954/1937] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:47 2022] DELETE /identity/v3/projects/2eb2250413fb4622b0e26f43e640a900 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:47.348583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c1c2c-b0fe-43a1-81d8-693bfe6a9e5c service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:47.373454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c1c2c-b0fe-43a1-81d8-693bfe6a9e5c service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:47.455312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:47.456208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:47.456488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:47.456763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] PATH_INFO: `/v3/users/1e6e9e17a8e442fa91e51856537c0e04` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:47.458674 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6e9e17a8e442fa91e51856537c0e04)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:47.460536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:47.514802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6e9e17a8e442fa91e51856537c0e04'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.528318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6e9e17a8e442fa91e51856537c0e04'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.538361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6e9e17a8e442fa91e51856537c0e04'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.539242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6e9e17a8e442fa91e51856537c0e04'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.540053 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6e9e17a8e442fa91e51856537c0e04'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:47.559723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:47.564640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 984/1938] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:47 2022] DELETE /identity/v3/users/1e6e9e17a8e442fa91e51856537c0e04 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:47.828509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:47.830129 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:47.851565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda1db0-6e75-4f21-afd6-a86d5ec1b260 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:47.936564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:47.937561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:47.938317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:47.938317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] PATH_INFO: `/v3/projects/d4175fb2ca824ebeb2dae1ec9396252c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:47.939608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4175fb2ca824ebeb2dae1ec9396252c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:47.941641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.007937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.009434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:48.014484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4175fb2ca824ebeb2dae1ec9396252c', 'request_id': 'req-5ee124f0-a0f1-4682-b78e-6bac2383f35b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.022909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4175fb2ca824ebeb2dae1ec9396252c', 'request_id': 'req-5ee124f0-a0f1-4682-b78e-6bac2383f35b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.033100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4175fb2ca824ebeb2dae1ec9396252c', 'request_id': 'req-5ee124f0-a0f1-4682-b78e-6bac2383f35b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.039036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80468b7-798d-42ab-b61c-d85a521febc4 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.048058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] The token cache is being invalidate because project d4175fb2ca824ebeb2dae1ec9396252c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:48.048973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.050410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.054946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 955/1939] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:47 2022] DELETE /identity/v3/projects/d4175fb2ca824ebeb2dae1ec9396252c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:48.056549 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.058140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:48.111164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.112043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.112326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.112634 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] PATH_INFO: `/v3/projects/43148bd16def484ba9aedcc95febec7c` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.113849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] RBAC: Authorizing `identity:delete_project(project_id=43148bd16def484ba9aedcc95febec7c)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.115915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.126822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee124f0-a0f1-4682-b78e-6bac2383f35b admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.131662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.132543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.132832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.133149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] PATH_INFO: `/v3/users/184efe9030864141bbff2d4a3d056266` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.134367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] RBAC: Authorizing `identity:delete_user(user_id=184efe9030864141bbff2d4a3d056266)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.137127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.145193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:48.174093 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43148bd16def484ba9aedcc95febec7c', 'request_id': 'req-6b93c788-2cad-42b9-9acd-47fb763a635e'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.181780 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43148bd16def484ba9aedcc95febec7c', 'request_id': 'req-6b93c788-2cad-42b9-9acd-47fb763a635e'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.193907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43148bd16def484ba9aedcc95febec7c', 'request_id': 'req-6b93c788-2cad-42b9-9acd-47fb763a635e'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.207082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184efe9030864141bbff2d4a3d056266'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.207996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184efe9030864141bbff2d4a3d056266'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.209380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] The token cache is being invalidate because project 43148bd16def484ba9aedcc95febec7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:48.209683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.210287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6b93c788-2cad-42b9-9acd-47fb763a635e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.212084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 985/1940] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:48 2022] DELETE /identity/v3/projects/43148bd16def484ba9aedcc95febec7c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:48.216234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184efe9030864141bbff2d4a3d056266'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.217302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184efe9030864141bbff2d4a3d056266'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.218366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032d527-1408-4d45-a244-7c9250f47472 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.218695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032d527-1408-4d45-a244-7c9250f47472 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.219865 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032d527-1408-4d45-a244-7c9250f47472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.233247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-990ff078-2ca0-4f61-9a10-8c5388cfb69b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184efe9030864141bbff2d4a3d056266'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.247562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d032d527-1408-4d45-a244-7c9250f47472 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:48.248979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 956/1941] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:48 2022] DELETE /identity/v3/users/184efe9030864141bbff2d4a3d056266 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:48.250865 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d032d527-1408-4d45-a244-7c9250f47472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:48.330487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 986/1942] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:48.341343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9821-a5cd-43cf-8445-45db0029299a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.341343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9821-a5cd-43cf-8445-45db0029299a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.341343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9821-a5cd-43cf-8445-45db0029299a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.363247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-784b9821-a5cd-43cf-8445-45db0029299a None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:48.364688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-784b9821-a5cd-43cf-8445-45db0029299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:48.376093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 957/1943] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:48.380405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032d527-1408-4d45-a244-7c9250f47472 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.390697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032d527-1408-4d45-a244-7c9250f47472 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.396311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf64f50-f5fe-4b35-9dcb-97afabb108a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.397102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf64f50-f5fe-4b35-9dcb-97afabb108a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.397388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf64f50-f5fe-4b35-9dcb-97afabb108a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.397665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf64f50-f5fe-4b35-9dcb-97afabb108a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.398398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf64f50-f5fe-4b35-9dcb-97afabb108a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:48.398708 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf64f50-f5fe-4b35-9dcb-97afabb108a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.400315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf64f50-f5fe-4b35-9dcb-97afabb108a6 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.413807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 987/1944] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:30:48.419267 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b9821-a5cd-43cf-8445-45db0029299a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.429470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b9821-a5cd-43cf-8445-45db0029299a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.434797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56cb90-73c2-4159-8f5a-8022f75c4fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.435473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56cb90-73c2-4159-8f5a-8022f75c4fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.435779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56cb90-73c2-4159-8f5a-8022f75c4fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.436215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56cb90-73c2-4159-8f5a-8022f75c4fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.436731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56cb90-73c2-4159-8f5a-8022f75c4fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.438710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56cb90-73c2-4159-8f5a-8022f75c4fc9 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.471966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 958/1945] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:48 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:48.476752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf64f50-f5fe-4b35-9dcb-97afabb108a6 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.488453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf64f50-f5fe-4b35-9dcb-97afabb108a6 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.508435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:48.566108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbdda9-c100-4f90-a0b5-e1ede42d4c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.566806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbdda9-c100-4f90-a0b5-e1ede42d4c7a admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.567195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbdda9-c100-4f90-a0b5-e1ede42d4c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.567482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbdda9-c100-4f90-a0b5-e1ede42d4c7a admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.568332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbdda9-c100-4f90-a0b5-e1ede42d4c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.570493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbdda9-c100-4f90-a0b5-e1ede42d4c7a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.601769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56cb90-73c2-4159-8f5a-8022f75c4fc9 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.604185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7b56cb90-73c2-4159-8f5a-8022f75c4fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:48.626710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56cb90-73c2-4159-8f5a-8022f75c4fc9 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.632610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.633569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.633853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.634126 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] PATH_INFO: `/v3/projects/8689ec9e1f43415699e0ed9be7dfa6ea` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.635688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 988/1946] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:48 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:48.637531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbdda9-c100-4f90-a0b5-e1ede42d4c7a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.638665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8689ec9e1f43415699e0ed9be7dfa6ea)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.641584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.655909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbdda9-c100-4f90-a0b5-e1ede42d4c7a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.665069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8e162-2e29-4752-88a2-73a6f3cf0ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.666053 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8e162-2e29-4752-88a2-73a6f3cf0ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.666333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8e162-2e29-4752-88a2-73a6f3cf0ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.666608 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8e162-2e29-4752-88a2-73a6f3cf0ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.668158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8e162-2e29-4752-88a2-73a6f3cf0ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.670510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8e162-2e29-4752-88a2-73a6f3cf0ca5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.685135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 989/1947] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:48.689215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8e162-2e29-4752-88a2-73a6f3cf0ca5 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.698718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8689ec9e1f43415699e0ed9be7dfa6ea', 'request_id': 'req-ebca6f47-ccc4-4611-aab1-04e14409fc6c'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.701369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8e162-2e29-4752-88a2-73a6f3cf0ca5 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.704811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8689ec9e1f43415699e0ed9be7dfa6ea', 'request_id': 'req-ebca6f47-ccc4-4611-aab1-04e14409fc6c'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.712981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8689ec9e1f43415699e0ed9be7dfa6ea', 'request_id': 'req-ebca6f47-ccc4-4611-aab1-04e14409fc6c'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.729113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] The token cache is being invalidate because project 8689ec9e1f43415699e0ed9be7dfa6ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:48.729724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.731153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ebca6f47-ccc4-4611-aab1-04e14409fc6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:48.732833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 959/1948] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:48 2022] DELETE /identity/v3/projects/8689ec9e1f43415699e0ed9be7dfa6ea => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:48.737525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102cd43-5953-41a7-82af-98247d4ff243 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.737886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102cd43-5953-41a7-82af-98247d4ff243 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.738241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102cd43-5953-41a7-82af-98247d4ff243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.759456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8102cd43-5953-41a7-82af-98247d4ff243 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:48.761375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8102cd43-5953-41a7-82af-98247d4ff243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:48.781513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52680dee-5d1d-4ca9-8223-8f1538d07ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.782314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52680dee-5d1d-4ca9-8223-8f1538d07ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.782596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52680dee-5d1d-4ca9-8223-8f1538d07ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.782871 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52680dee-5d1d-4ca9-8223-8f1538d07ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.783603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52680dee-5d1d-4ca9-8223-8f1538d07ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.785298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52680dee-5d1d-4ca9-8223-8f1538d07ac2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.796444 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 990/1949] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:30:48.796783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:48.800622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52680dee-5d1d-4ca9-8223-8f1538d07ac2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.802915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-52680dee-5d1d-4ca9-8223-8f1538d07ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:48.815687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52680dee-5d1d-4ca9-8223-8f1538d07ac2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.820418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae07db-34e0-421a-aab7-8da03716ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.821331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae07db-34e0-421a-aab7-8da03716ea2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.821612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae07db-34e0-421a-aab7-8da03716ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.821886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae07db-34e0-421a-aab7-8da03716ea2a admin admin] PATH_INFO: `/v3/projects/7a9e26fa74704be18c95c9572714c4f6/users/86266900bbfe449aaf6c78027859644b/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.828665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 960/1950] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:48.836386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a7029-1918-4467-84ad-8c5f5f5b1659 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.836672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a7029-1918-4467-84ad-8c5f5f5b1659 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.836976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a7029-1918-4467-84ad-8c5f5f5b1659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.837530 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae07db-34e0-421a-aab7-8da03716ea2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9e26fa74704be18c95c9572714c4f6, user_id=86266900bbfe449aaf6c78027859644b, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.841036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae07db-34e0-421a-aab7-8da03716ea2a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.857285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-873a7029-1918-4467-84ad-8c5f5f5b1659 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:48.858514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-873a7029-1918-4467-84ad-8c5f5f5b1659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:48.862113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 991/1951] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:48 2022] PUT /identity/v3/projects/7a9e26fa74704be18c95c9572714c4f6/users/86266900bbfe449aaf6c78027859644b/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:30:48.873889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d01024-245c-46a3-9ea5-1532db07c6c3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.874196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d01024-245c-46a3-9ea5-1532db07c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.874472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d01024-245c-46a3-9ea5-1532db07c6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.901583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-41d01024-245c-46a3-9ea5-1532db07c6c3 None None] MFA Rules not processed for user `86266900bbfe449aaf6c78027859644b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:48.903402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-41d01024-245c-46a3-9ea5-1532db07c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:48.920152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 961/1952] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:48.924377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a7029-1918-4467-84ad-8c5f5f5b1659 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.933810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a7029-1918-4467-84ad-8c5f5f5b1659 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.938413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ff210-0db9-4d8d-86e8-34a0ee81c706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.939063 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ff210-0db9-4d8d-86e8-34a0ee81c706 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.939340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ff210-0db9-4d8d-86e8-34a0ee81c706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.939614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ff210-0db9-4d8d-86e8-34a0ee81c706 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.940328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ff210-0db9-4d8d-86e8-34a0ee81c706 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:48.940633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ff210-0db9-4d8d-86e8-34a0ee81c706 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.942293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ff210-0db9-4d8d-86e8-34a0ee81c706 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.957437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 962/1953] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:30:48.958390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ff210-0db9-4d8d-86e8-34a0ee81c706 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.964322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:48.969144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ff210-0db9-4d8d-86e8-34a0ee81c706 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:48.971001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 992/1954] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:48.979564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69202d93-a222-4ecb-9ad5-c476edbd14ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:48.980888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69202d93-a222-4ecb-9ad5-c476edbd14ac admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:48.981187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69202d93-a222-4ecb-9ad5-c476edbd14ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:48.981461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69202d93-a222-4ecb-9ad5-c476edbd14ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:48.982150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69202d93-a222-4ecb-9ad5-c476edbd14ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:48.983724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d01024-245c-46a3-9ea5-1532db07c6c3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:48.984845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69202d93-a222-4ecb-9ad5-c476edbd14ac admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:48.985695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-41d01024-245c-46a3-9ea5-1532db07c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:49.023379 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 963/1955] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:48 2022] POST /identity/v3/projects => generated 359 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:49.028277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69202d93-a222-4ecb-9ad5-c476edbd14ac admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:49.061870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d01024-245c-46a3-9ea5-1532db07c6c3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:49.112607 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69202d93-a222-4ecb-9ad5-c476edbd14ac admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:49.120305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a8c48-a422-4e5f-823c-172caaf48c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:49.121733 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a8c48-a422-4e5f-823c-172caaf48c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:49.121922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a8c48-a422-4e5f-823c-172caaf48c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:49.122198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a8c48-a422-4e5f-823c-172caaf48c61 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:49.122965 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a8c48-a422-4e5f-823c-172caaf48c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:49.125817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a8c48-a422-4e5f-823c-172caaf48c61 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:49.130314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909086a7-7ad3-4953-a428-6d8c79150362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:49.130981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909086a7-7ad3-4953-a428-6d8c79150362 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:49.131270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909086a7-7ad3-4953-a428-6d8c79150362 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:49.131548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909086a7-7ad3-4953-a428-6d8c79150362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:49.160852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 964/1956] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:49 2022] POST /identity/v3/users => generated 280 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:49.165017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a8c48-a422-4e5f-823c-172caaf48c61 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:49.177204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a8c48-a422-4e5f-823c-172caaf48c61 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:49.182952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889d488-b6d5-4750-b1fc-f41694fc8fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:49.183994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889d488-b6d5-4750-b1fc-f41694fc8fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:49.184160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889d488-b6d5-4750-b1fc-f41694fc8fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:49.184967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889d488-b6d5-4750-b1fc-f41694fc8fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:49.185770 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889d488-b6d5-4750-b1fc-f41694fc8fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:49.187425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889d488-b6d5-4750-b1fc-f41694fc8fa3 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:49.198200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909086a7-7ad3-4953-a428-6d8c79150362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:49.201134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 965/1957] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:49.202448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909086a7-7ad3-4953-a428-6d8c79150362 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:49.202593 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889d488-b6d5-4750-b1fc-f41694fc8fa3 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:49.212370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 993/1958] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:30:49.214898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889d488-b6d5-4750-b1fc-f41694fc8fa3 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:49.219900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8d232-a6bd-4159-916f-7a48f9ce0b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:49.220553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8d232-a6bd-4159-916f-7a48f9ce0b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:49.220840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8d232-a6bd-4159-916f-7a48f9ce0b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:49.221150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8d232-a6bd-4159-916f-7a48f9ce0b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:49.221894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8d232-a6bd-4159-916f-7a48f9ce0b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:49.223493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8d232-a6bd-4159-916f-7a48f9ce0b1b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:49.235436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 966/1959] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:49.239640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909086a7-7ad3-4953-a428-6d8c79150362 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:49.252975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909086a7-7ad3-4953-a428-6d8c79150362 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:49.257858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be42497-1e75-4aff-97b0-858d99524078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:49.258753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be42497-1e75-4aff-97b0-858d99524078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:49.259217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be42497-1e75-4aff-97b0-858d99524078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:49.259589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be42497-1e75-4aff-97b0-858d99524078 admin admin] PATH_INFO: `/v3/projects/63fa0e3787e94207a6944c7f5f202329/users/a60344f7e0fb4196a02f38fce29e8a4e/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:49.278069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be42497-1e75-4aff-97b0-858d99524078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fa0e3787e94207a6944c7f5f202329, user_id=a60344f7e0fb4196a02f38fce29e8a4e, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:49.281019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be42497-1e75-4aff-97b0-858d99524078 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:49.294792 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 994/1960] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:49 2022] PUT /identity/v3/projects/63fa0e3787e94207a6944c7f5f202329/users/a60344f7e0fb4196a02f38fce29e8a4e/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:49.307332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bf2b5-6991-4359-bf77-768973df5c96 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:49.307650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bf2b5-6991-4359-bf77-768973df5c96 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:49.308594 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bf2b5-6991-4359-bf77-768973df5c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:49.335141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:49.338845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8c6bf2b5-6991-4359-bf77-768973df5c96 None None] MFA Rules not processed for user `a60344f7e0fb4196a02f38fce29e8a4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:49.342729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8c6bf2b5-6991-4359-bf77-768973df5c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:49.438693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 967/1961] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:49.539471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be42497-1e75-4aff-97b0-858d99524078 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:49.544084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3be42497-1e75-4aff-97b0-858d99524078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:49.650001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:49.692295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be42497-1e75-4aff-97b0-858d99524078 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:49.704327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dac3b3-56fa-477a-b58c-56ea592792d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:49.705028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac3b3-56fa-477a-b58c-56ea592792d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:49.705314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac3b3-56fa-477a-b58c-56ea592792d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:49.705591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac3b3-56fa-477a-b58c-56ea592792d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:49.726136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dac3b3-56fa-477a-b58c-56ea592792d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:49.727618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dac3b3-56fa-477a-b58c-56ea592792d0 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:49.755041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 995/1962] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:49 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:49.774853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bf2b5-6991-4359-bf77-768973df5c96 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:49.883181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bf2b5-6991-4359-bf77-768973df5c96 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:49.888196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:49.889184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:49.889543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:49.889927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] PATH_INFO: `/v3/users/a60344f7e0fb4196a02f38fce29e8a4e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:49.891189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60344f7e0fb4196a02f38fce29e8a4e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:49.893534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:49.947985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60344f7e0fb4196a02f38fce29e8a4e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:49.948682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60344f7e0fb4196a02f38fce29e8a4e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:49.956636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60344f7e0fb4196a02f38fce29e8a4e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:49.959504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60344f7e0fb4196a02f38fce29e8a4e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:49.978654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60344f7e0fb4196a02f38fce29e8a4e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:49.996331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 968/1963] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:30:49 2022] DELETE /identity/v3/users/a60344f7e0fb4196a02f38fce29e8a4e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:50.001354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3202df-5cea-4d85-9ed9-dfd695cecf39 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:50.002087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3202df-5cea-4d85-9ed9-dfd695cecf39 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:50.002087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3202df-5cea-4d85-9ed9-dfd695cecf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:50.023095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-dc3202df-5cea-4d85-9ed9-dfd695cecf39 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:50.024300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-dc3202df-5cea-4d85-9ed9-dfd695cecf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:50.086357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 996/1964] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:30:50.096833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:50.098203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:50.124791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:50.158208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e874f8-1003-4f34-9f65-6dd1c6a28c8d admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:50.162246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea38234-83ba-4786-a69c-ffe60940ac26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:50.162893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea38234-83ba-4786-a69c-ffe60940ac26 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:50.163178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea38234-83ba-4786-a69c-ffe60940ac26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:50.163457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea38234-83ba-4786-a69c-ffe60940ac26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:50.172784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea38234-83ba-4786-a69c-ffe60940ac26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:50.174292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea38234-83ba-4786-a69c-ffe60940ac26 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:50.184713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 969/1965] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:30:50.458032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3202df-5cea-4d85-9ed9-dfd695cecf39 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:50.459599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-dc3202df-5cea-4d85-9ed9-dfd695cecf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:50.474620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3202df-5cea-4d85-9ed9-dfd695cecf39 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:50.483944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:50.486761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:50.487349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:50.487877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] PATH_INFO: `/v3/projects/63fa0e3787e94207a6944c7f5f202329` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:50.490071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] RBAC: Authorizing `identity:delete_project(project_id=63fa0e3787e94207a6944c7f5f202329)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:50.494088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:50.593366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63fa0e3787e94207a6944c7f5f202329', 'request_id': 'req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:50.608919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63fa0e3787e94207a6944c7f5f202329', 'request_id': 'req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:50.621226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63fa0e3787e94207a6944c7f5f202329', 'request_id': 'req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:50.644526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] The token cache is being invalidate because project 63fa0e3787e94207a6944c7f5f202329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:30:50.644829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:50.645587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4a10c9d0-3657-484e-bf0b-434ef1c6b05f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:30:50.647333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 997/1966] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:30:50 2022] DELETE /identity/v3/projects/63fa0e3787e94207a6944c7f5f202329 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:50.653154 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b9769-4eac-46e4-ac73-b96e2cb22f7d None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:50.653993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b9769-4eac-46e4-ac73-b96e2cb22f7d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:50.654279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b9769-4eac-46e4-ac73-b96e2cb22f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:50.676380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-690b9769-4eac-46e4-ac73-b96e2cb22f7d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:50.677622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-690b9769-4eac-46e4-ac73-b96e2cb22f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:50.764075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 970/1967] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:50.771391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a488-dea3-4056-8057-3257a3902c71 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:50.771717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a488-dea3-4056-8057-3257a3902c71 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:50.771993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a488-dea3-4056-8057-3257a3902c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:50.795168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1ce1a488-dea3-4056-8057-3257a3902c71 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:50.796781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1ce1a488-dea3-4056-8057-3257a3902c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:50.811969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 998/1968] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:50.813186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b9769-4eac-46e4-ac73-b96e2cb22f7d None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:50.826895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b9769-4eac-46e4-ac73-b96e2cb22f7d None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:50.836779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1c66-97a1-4b2f-aa79-6f6715379438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:50.836779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1c66-97a1-4b2f-aa79-6f6715379438 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:50.836779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1c66-97a1-4b2f-aa79-6f6715379438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:50.836779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1c66-97a1-4b2f-aa79-6f6715379438 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:50.837936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa1c66-97a1-4b2f-aa79-6f6715379438 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:30:50.840654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa1c66-97a1-4b2f-aa79-6f6715379438 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:50.843995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa1c66-97a1-4b2f-aa79-6f6715379438 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:50.883968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 971/1969] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:30:50.890165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a488-dea3-4056-8057-3257a3902c71 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:50.903410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a488-dea3-4056-8057-3257a3902c71 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:50.910226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5db22-ef90-4355-acac-6cdf5ef9248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:50.913091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5db22-ef90-4355-acac-6cdf5ef9248c admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:50.913425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5db22-ef90-4355-acac-6cdf5ef9248c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:50.913690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5db22-ef90-4355-acac-6cdf5ef9248c admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:50.914541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5db22-ef90-4355-acac-6cdf5ef9248c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:50.916586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5db22-ef90-4355-acac-6cdf5ef9248c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:50.955747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 999/1970] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:30:50 2022] POST /identity/v3/projects => generated 389 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:30:50.960469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1c66-97a1-4b2f-aa79-6f6715379438 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:50.985950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1c66-97a1-4b2f-aa79-6f6715379438 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:51.058192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9bcce-7397-4168-8379-44bc24922266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:51.059583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9bcce-7397-4168-8379-44bc24922266 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:51.060159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9bcce-7397-4168-8379-44bc24922266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:51.060509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9bcce-7397-4168-8379-44bc24922266 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:51.061408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9bcce-7397-4168-8379-44bc24922266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:51.064289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9bcce-7397-4168-8379-44bc24922266 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:51.096397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 972/1971] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:30:50 2022] POST /identity/v3/users => generated 295 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:30:51.100490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5db22-ef90-4355-acac-6cdf5ef9248c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:51.112600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5db22-ef90-4355-acac-6cdf5ef9248c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:51.117531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e432b-9393-4efb-a102-c3cb1c54f9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:51.118305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e432b-9393-4efb-a102-c3cb1c54f9fe admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:51.118659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e432b-9393-4efb-a102-c3cb1c54f9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:51.119483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e432b-9393-4efb-a102-c3cb1c54f9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:51.119796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e432b-9393-4efb-a102-c3cb1c54f9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:51.121562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e432b-9393-4efb-a102-c3cb1c54f9fe admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:51.134716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1000/1972] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:51.140046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9bcce-7397-4168-8379-44bc24922266 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:51.155578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9bcce-7397-4168-8379-44bc24922266 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:51.161308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d8e43-7411-47a4-aec0-b0535be31edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:51.162100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d8e43-7411-47a4-aec0-b0535be31edd admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:51.162483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d8e43-7411-47a4-aec0-b0535be31edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:51.162831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d8e43-7411-47a4-aec0-b0535be31edd admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:51.163630 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d8e43-7411-47a4-aec0-b0535be31edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:51.165367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d8e43-7411-47a4-aec0-b0535be31edd admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:51.183964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 973/1973] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:30:51.185961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e432b-9393-4efb-a102-c3cb1c54f9fe admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:51.196342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e432b-9393-4efb-a102-c3cb1c54f9fe admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:51.204034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f4a82-f8af-4d4a-bd10-2099765af0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:51.204335 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f4a82-f8af-4d4a-bd10-2099765af0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:51.204622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f4a82-f8af-4d4a-bd10-2099765af0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:51.204958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f4a82-f8af-4d4a-bd10-2099765af0d7 admin admin] PATH_INFO: `/v3/projects/5bc7e3b869ea4ac39e61c50f86e2cd6b/users/e76707e47de34c19a093b5c5d145d17a/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:51.227525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f4a82-f8af-4d4a-bd10-2099765af0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc7e3b869ea4ac39e61c50f86e2cd6b, user_id=e76707e47de34c19a093b5c5d145d17a, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:51.231681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f4a82-f8af-4d4a-bd10-2099765af0d7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:51.247393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1001/1974] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:30:51 2022] PUT /identity/v3/projects/5bc7e3b869ea4ac39e61c50f86e2cd6b/users/e76707e47de34c19a093b5c5d145d17a/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:30:51.257868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c35ac2-6dc0-4d4a-b357-12ed4e7a432f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:51.258378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c35ac2-6dc0-4d4a-b357-12ed4e7a432f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:51.259806 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c35ac2-6dc0-4d4a-b357-12ed4e7a432f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:51.295568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-40c35ac2-6dc0-4d4a-b357-12ed4e7a432f None None] MFA Rules not processed for user `e76707e47de34c19a093b5c5d145d17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:30:51.297237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-40c35ac2-6dc0-4d4a-b357-12ed4e7a432f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:51.321668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:51.380461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 974/1975] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:30:51.391296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f4a82-f8af-4d4a-bd10-2099765af0d7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:30:51.393046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0f0f4a82-f8af-4d4a-bd10-2099765af0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:30:51.403830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:30:51.465109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f4a82-f8af-4d4a-bd10-2099765af0d7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:30:51.469427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c00e58-ff67-4acd-ab1a-785a2db29a17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:30:51.470181 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c00e58-ff67-4acd-ab1a-785a2db29a17 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:30:51.470972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c00e58-ff67-4acd-ab1a-785a2db29a17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:30:51.470972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c00e58-ff67-4acd-ab1a-785a2db29a17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:30:51.483305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c00e58-ff67-4acd-ab1a-785a2db29a17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:30:51.485635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c00e58-ff67-4acd-ab1a-785a2db29a17 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:30:51.495900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1002/1976] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:30:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:01.376268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=18 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:01.491133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=23 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:11.659677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad25ce-9ebe-4880-a0b2-cec9001a6c24 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:11.660590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad25ce-9ebe-4880-a0b2-cec9001a6c24 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:11.660895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad25ce-9ebe-4880-a0b2-cec9001a6c24 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:11.662556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 975/1977] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:31:11.672904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad97b-ba93-45c0-aee9-301b85844a34 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:11.673433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad97b-ba93-45c0-aee9-301b85844a34 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:11.673718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad97b-ba93-45c0-aee9-301b85844a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:11.703411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-900ad97b-ba93-45c0-aee9-301b85844a34 None None] MFA Rules not processed for user `3a7dd5a0102d4d3493f09e809cb3caaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:11.705335 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-900ad97b-ba93-45c0-aee9-301b85844a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:11.741375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad25ce-9ebe-4880-a0b2-cec9001a6c24 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:11.743089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8bad25ce-9ebe-4880-a0b2-cec9001a6c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:11.811647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1003/1978] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:31:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:11.823104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ad97b-ba93-45c0-aee9-301b85844a34 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:11.825477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-900ad97b-ba93-45c0-aee9-301b85844a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:11.835957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad25ce-9ebe-4880-a0b2-cec9001a6c24 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:11.841999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:11.842879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:11.843163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:11.843443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] PATH_INFO: `/v3/users/e91e38a712074ca5a2567da012445aa3` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:11.843549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ad97b-ba93-45c0-aee9-301b85844a34 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:11.844648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e91e38a712074ca5a2567da012445aa3)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:11.846666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:11.848777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b552f63-8740-471a-8802-e8b99fa4b3e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:11.849492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b552f63-8740-471a-8802-e8b99fa4b3e5 service placement] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:11.849778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b552f63-8740-471a-8802-e8b99fa4b3e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:11.850177 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b552f63-8740-471a-8802-e8b99fa4b3e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:11.859592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b552f63-8740-471a-8802-e8b99fa4b3e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:11.861313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b552f63-8740-471a-8802-e8b99fa4b3e5 service placement] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:11.871982 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1004/1979] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:11.897599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e91e38a712074ca5a2567da012445aa3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:11.898262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e91e38a712074ca5a2567da012445aa3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:11.906015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e91e38a712074ca5a2567da012445aa3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:11.906713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e91e38a712074ca5a2567da012445aa3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:11.919810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-bd45c2ee-78b4-499b-b184-bbf12b96029c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e91e38a712074ca5a2567da012445aa3'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:11.932476 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 976/1980] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:11 2022] DELETE /identity/v3/users/e91e38a712074ca5a2567da012445aa3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:12.110309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b552f63-8740-471a-8802-e8b99fa4b3e5 service placement] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:12.121748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b552f63-8740-471a-8802-e8b99fa4b3e5 service placement] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:12.197534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:12.198430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.198700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.199112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] PATH_INFO: `/v3/projects/377219e53ce94ab089e48437d3468663` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.200425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=377219e53ce94ab089e48437d3468663)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:12.202333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:12.254683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '377219e53ce94ab089e48437d3468663', 'request_id': 'req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:12.261159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '377219e53ce94ab089e48437d3468663', 'request_id': 'req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:12.270006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '377219e53ce94ab089e48437d3468663', 'request_id': 'req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:12.284829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] The token cache is being invalidate because project 377219e53ce94ab089e48437d3468663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:12.285159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:12.285815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ad1f2f3-d77c-46d3-a9bf-01d91d2c1691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:12.287397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1005/1981] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:12 2022] DELETE /identity/v3/projects/377219e53ce94ab089e48437d3468663 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:12.292558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef692c-ddfd-44d9-b1c1-82ba94bacbf9 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.293678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef692c-ddfd-44d9-b1c1-82ba94bacbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.293945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef692c-ddfd-44d9-b1c1-82ba94bacbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.312597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-c1ef692c-ddfd-44d9-b1c1-82ba94bacbf9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:12.313971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c1ef692c-ddfd-44d9-b1c1-82ba94bacbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:12.377424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 977/1982] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:12.385126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82329ff-809d-4e75-8cb2-77b56f995d22 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.385439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82329ff-809d-4e75-8cb2-77b56f995d22 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.385745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82329ff-809d-4e75-8cb2-77b56f995d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.405391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-e82329ff-809d-4e75-8cb2-77b56f995d22 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:12.406604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e82329ff-809d-4e75-8cb2-77b56f995d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:12.414685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1006/1983] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:12.418937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef692c-ddfd-44d9-b1c1-82ba94bacbf9 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:12.431578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef692c-ddfd-44d9-b1c1-82ba94bacbf9 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:12.437724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fea99-a5bd-48c3-9a2f-2fb066ece4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:12.438389 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fea99-a5bd-48c3-9a2f-2fb066ece4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.438669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fea99-a5bd-48c3-9a2f-2fb066ece4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.438943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fea99-a5bd-48c3-9a2f-2fb066ece4f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.439673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fea99-a5bd-48c3-9a2f-2fb066ece4f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:12.439979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fea99-a5bd-48c3-9a2f-2fb066ece4f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:12.441626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fea99-a5bd-48c3-9a2f-2fb066ece4f8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:12.456362 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 978/1984] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:12.460065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82329ff-809d-4e75-8cb2-77b56f995d22 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:12.471910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82329ff-809d-4e75-8cb2-77b56f995d22 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:12.476826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8929b2d-cc02-4cc9-9553-10044428f863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:12.477527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8929b2d-cc02-4cc9-9553-10044428f863 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.477810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8929b2d-cc02-4cc9-9553-10044428f863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.478085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8929b2d-cc02-4cc9-9553-10044428f863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.478777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8929b2d-cc02-4cc9-9553-10044428f863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:12.481105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8929b2d-cc02-4cc9-9553-10044428f863 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:12.485048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:12.522047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1007/1985] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:12 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:12.526076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fea99-a5bd-48c3-9a2f-2fb066ece4f8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:12.537977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fea99-a5bd-48c3-9a2f-2fb066ece4f8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:12.544584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:12.611381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c30589-83f3-45c9-8b31-dbe2155a558f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:12.612494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c30589-83f3-45c9-8b31-dbe2155a558f admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.612494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c30589-83f3-45c9-8b31-dbe2155a558f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.612645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c30589-83f3-45c9-8b31-dbe2155a558f admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.613301 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c30589-83f3-45c9-8b31-dbe2155a558f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:12.615339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c30589-83f3-45c9-8b31-dbe2155a558f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:12.644907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 979/1986] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:12 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:12.647591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8929b2d-cc02-4cc9-9553-10044428f863 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:12.661448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8929b2d-cc02-4cc9-9553-10044428f863 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:12.669069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f7901-9129-493e-aab8-88888624ec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:12.669069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f7901-9129-493e-aab8-88888624ec25 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.669069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f7901-9129-493e-aab8-88888624ec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.669069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f7901-9129-493e-aab8-88888624ec25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.669374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f7901-9129-493e-aab8-88888624ec25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:12.670510 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f7901-9129-493e-aab8-88888624ec25 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:12.684150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1008/1987] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:12.688483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c30589-83f3-45c9-8b31-dbe2155a558f admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:12.702399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c30589-83f3-45c9-8b31-dbe2155a558f admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:12.709004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dda7c3-e4a2-444c-b716-66c6bc27aac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:12.710085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dda7c3-e4a2-444c-b716-66c6bc27aac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.710969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dda7c3-e4a2-444c-b716-66c6bc27aac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.711247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dda7c3-e4a2-444c-b716-66c6bc27aac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.711983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dda7c3-e4a2-444c-b716-66c6bc27aac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:12.713668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dda7c3-e4a2-444c-b716-66c6bc27aac8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:12.725536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 980/1988] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:12.729592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f7901-9129-493e-aab8-88888624ec25 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:12.742010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f7901-9129-493e-aab8-88888624ec25 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:12.749515 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb7f9-9ca3-4f9c-9d62-818729c75147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:12.750431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb7f9-9ca3-4f9c-9d62-818729c75147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.750719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb7f9-9ca3-4f9c-9d62-818729c75147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.751000 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb7f9-9ca3-4f9c-9d62-818729c75147 admin admin] PATH_INFO: `/v3/projects/934e97feb41542908e28114363996e77/users/a6c6dd9e7b304057a076420e7c09df90/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.770473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cb7f9-9ca3-4f9c-9d62-818729c75147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934e97feb41542908e28114363996e77, user_id=a6c6dd9e7b304057a076420e7c09df90, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:12.773172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cb7f9-9ca3-4f9c-9d62-818729c75147 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:12.789740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1009/1989] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:12 2022] PUT /identity/v3/projects/934e97feb41542908e28114363996e77/users/a6c6dd9e7b304057a076420e7c09df90/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:12.801040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f9760-8b05-4e88-9f31-d536c701d921 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:12.801330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f9760-8b05-4e88-9f31-d536c701d921 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:12.801832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f9760-8b05-4e88-9f31-d536c701d921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:12.850200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4b8f9760-8b05-4e88-9f31-d536c701d921 None None] MFA Rules not processed for user `a6c6dd9e7b304057a076420e7c09df90`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:12.850585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4b8f9760-8b05-4e88-9f31-d536c701d921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:12.961070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 981/1990] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:12.966761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb7f9-9ca3-4f9c-9d62-818729c75147 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.035980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb7f9-9ca3-4f9c-9d62-818729c75147 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.040750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264040d6-1f72-4371-ac47-2e913ea127ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.041446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264040d6-1f72-4371-ac47-2e913ea127ba admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.041730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264040d6-1f72-4371-ac47-2e913ea127ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.042018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264040d6-1f72-4371-ac47-2e913ea127ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.042849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264040d6-1f72-4371-ac47-2e913ea127ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.045286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264040d6-1f72-4371-ac47-2e913ea127ba admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.074647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1010/1991] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:12 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:13.075885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f9760-8b05-4e88-9f31-d536c701d921 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.091712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f9760-8b05-4e88-9f31-d536c701d921 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.149432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a76f39-356b-41af-aa57-20bfa600d5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.149993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a76f39-356b-41af-aa57-20bfa600d5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.150274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a76f39-356b-41af-aa57-20bfa600d5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.150578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a76f39-356b-41af-aa57-20bfa600d5aa admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.151265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a76f39-356b-41af-aa57-20bfa600d5aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.153979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a76f39-356b-41af-aa57-20bfa600d5aa admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.178260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 982/1992] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:13 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:13.184668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264040d6-1f72-4371-ac47-2e913ea127ba admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.196278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264040d6-1f72-4371-ac47-2e913ea127ba admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.201448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528f947-b03c-4aa6-80b1-f16aeddcf15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.202122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528f947-b03c-4aa6-80b1-f16aeddcf15b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.202434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528f947-b03c-4aa6-80b1-f16aeddcf15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.202711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528f947-b03c-4aa6-80b1-f16aeddcf15b admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.203444 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528f947-b03c-4aa6-80b1-f16aeddcf15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.205058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528f947-b03c-4aa6-80b1-f16aeddcf15b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.217409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1011/1993] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:13.221573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a76f39-356b-41af-aa57-20bfa600d5aa admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.228066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:13.233158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a76f39-356b-41af-aa57-20bfa600d5aa admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.241711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d36a5-3a19-4daf-934d-113a4230a1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.242613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d36a5-3a19-4daf-934d-113a4230a1a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.242898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d36a5-3a19-4daf-934d-113a4230a1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.243248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d36a5-3a19-4daf-934d-113a4230a1a5 admin admin] PATH_INFO: `/v3/projects/d13bf39a43624a6d9dfa2d8232138307/users/fdf25b7a8bc343b0a15e4c9be71df4be/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.258768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d36a5-3a19-4daf-934d-113a4230a1a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d13bf39a43624a6d9dfa2d8232138307, user_id=fdf25b7a8bc343b0a15e4c9be71df4be, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.261870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d36a5-3a19-4daf-934d-113a4230a1a5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.275774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 983/1994] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:13 2022] PUT /identity/v3/projects/d13bf39a43624a6d9dfa2d8232138307/users/fdf25b7a8bc343b0a15e4c9be71df4be/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:13.280436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528f947-b03c-4aa6-80b1-f16aeddcf15b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.290662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528f947-b03c-4aa6-80b1-f16aeddcf15b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.351419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55563062-4e46-42a3-90c3-1dc75d7d740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.352137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55563062-4e46-42a3-90c3-1dc75d7d740e admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.352420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55563062-4e46-42a3-90c3-1dc75d7d740e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.352712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55563062-4e46-42a3-90c3-1dc75d7d740e admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.353598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55563062-4e46-42a3-90c3-1dc75d7d740e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.355225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55563062-4e46-42a3-90c3-1dc75d7d740e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.366356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1012/1995] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:13.370511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d36a5-3a19-4daf-934d-113a4230a1a5 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.380739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d36a5-3a19-4daf-934d-113a4230a1a5 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.385513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505e078-c621-420c-8028-6779beafda92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.387227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505e078-c621-420c-8028-6779beafda92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.387528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505e078-c621-420c-8028-6779beafda92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.387873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505e078-c621-420c-8028-6779beafda92 admin admin] PATH_INFO: `/v3/projects/d13bf39a43624a6d9dfa2d8232138307/users/fdf25b7a8bc343b0a15e4c9be71df4be/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.390004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505e078-c621-420c-8028-6779beafda92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d13bf39a43624a6d9dfa2d8232138307, user_id=fdf25b7a8bc343b0a15e4c9be71df4be, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.392785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505e078-c621-420c-8028-6779beafda92 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.405085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 984/1996] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:13 2022] PUT /identity/v3/projects/d13bf39a43624a6d9dfa2d8232138307/users/fdf25b7a8bc343b0a15e4c9be71df4be/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:13.415128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a703f-ffba-4c7d-8068-1f5ed9636f27 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.415509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a703f-ffba-4c7d-8068-1f5ed9636f27 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.415783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a703f-ffba-4c7d-8068-1f5ed9636f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.423301 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020285d-9f19-4d0c-8eeb-73f6e6197844 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.424208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020285d-9f19-4d0c-8eeb-73f6e6197844 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.424891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020285d-9f19-4d0c-8eeb-73f6e6197844 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.427265 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 985/1997] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:31:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:31:13.444317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f87a703f-ffba-4c7d-8068-1f5ed9636f27 None None] MFA Rules not processed for user `fdf25b7a8bc343b0a15e4c9be71df4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:13.445568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f87a703f-ffba-4c7d-8068-1f5ed9636f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:13.452905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9027df9-7108-4c5b-be0a-f88ff7f0488c None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.453062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9027df9-7108-4c5b-be0a-f88ff7f0488c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.453259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9027df9-7108-4c5b-be0a-f88ff7f0488c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.455592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:13.474133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e9027df9-7108-4c5b-be0a-f88ff7f0488c None None] MFA Rules not processed for user `3a7dd5a0102d4d3493f09e809cb3caaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:13.475326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e9027df9-7108-4c5b-be0a-f88ff7f0488c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:13.530990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1013/1998] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:13.534694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a703f-ffba-4c7d-8068-1f5ed9636f27 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.537241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f87a703f-ffba-4c7d-8068-1f5ed9636f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:13.549895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 986/1999] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:13.550811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9027df9-7108-4c5b-be0a-f88ff7f0488c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.552174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e9027df9-7108-4c5b-be0a-f88ff7f0488c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:13.608279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a703f-ffba-4c7d-8068-1f5ed9636f27 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.613668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9027df9-7108-4c5b-be0a-f88ff7f0488c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.614678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.615565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.615850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.616126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] PATH_INFO: `/v3/users/767b5629a2ce472cb57ebc2421597338` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.617439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] RBAC: Authorizing `identity:delete_user(user_id=767b5629a2ce472cb57ebc2421597338)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.617587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31e918-4db3-4bf5-a822-f416e05e5568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.618264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31e918-4db3-4bf5-a822-f416e05e5568 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.618547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31e918-4db3-4bf5-a822-f416e05e5568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.618822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31e918-4db3-4bf5-a822-f416e05e5568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.619463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.628620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31e918-4db3-4bf5-a822-f416e05e5568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.630159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31e918-4db3-4bf5-a822-f416e05e5568 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.642036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 987/2000] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:13.643952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31e918-4db3-4bf5-a822-f416e05e5568 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.645309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3e31e918-4db3-4bf5-a822-f416e05e5568 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:13.663038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31e918-4db3-4bf5-a822-f416e05e5568 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.668302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ebebc-9562-40d4-ac8f-af443e5fb896 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.669127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ebebc-9562-40d4-ac8f-af443e5fb896 service placement] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.669407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ebebc-9562-40d4-ac8f-af443e5fb896 service placement] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.669682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ebebc-9562-40d4-ac8f-af443e5fb896 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.673113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '767b5629a2ce472cb57ebc2421597338'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:13.678222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ebebc-9562-40d4-ac8f-af443e5fb896 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.679830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ebebc-9562-40d4-ac8f-af443e5fb896 service placement] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.688731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '767b5629a2ce472cb57ebc2421597338'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:13.691106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 988/2001] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:13.693299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ebebc-9562-40d4-ac8f-af443e5fb896 service placement] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.694824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-682ebebc-9562-40d4-ac8f-af443e5fb896 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:13.697938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '767b5629a2ce472cb57ebc2421597338'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:13.698705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '767b5629a2ce472cb57ebc2421597338'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:13.700011 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '767b5629a2ce472cb57ebc2421597338'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:13.709052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ebebc-9562-40d4-ac8f-af443e5fb896 service placement] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:13.713520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1ca1f-e2af-4158-b7d9-76fb9b6bc3eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:13.714252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1ca1f-e2af-4158-b7d9-76fb9b6bc3eb service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:13.714616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1ca1f-e2af-4158-b7d9-76fb9b6bc3eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:13.716282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1014/2002] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:13 2022] DELETE /identity/v3/users/767b5629a2ce472cb57ebc2421597338 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:13.717003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1ca1f-e2af-4158-b7d9-76fb9b6bc3eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:13.718967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-94f1ca1f-e2af-4158-b7d9-76fb9b6bc3eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:13.775248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:13.787227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1ca1f-e2af-4158-b7d9-76fb9b6bc3eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:13.789455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1ca1f-e2af-4158-b7d9-76fb9b6bc3eb service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:13.799819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 989/2003] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:31:13.901210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:13.904115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:13.938816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:14.004659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cb33d-3d6b-4dab-9b6f-2fdb4187114a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:14.009609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:14.010480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.010768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.011096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] PATH_INFO: `/v3/projects/d112fea2e24f469a8d54a7950aa74862` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.012779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d112fea2e24f469a8d54a7950aa74862)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:14.014808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:14.092582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd112fea2e24f469a8d54a7950aa74862', 'request_id': 'req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.092582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd112fea2e24f469a8d54a7950aa74862', 'request_id': 'req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.097692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd112fea2e24f469a8d54a7950aa74862', 'request_id': 'req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.114879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] The token cache is being invalidate because project d112fea2e24f469a8d54a7950aa74862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:14.115095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.115754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.120287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1015/2004] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:13 2022] DELETE /identity/v3/projects/d112fea2e24f469a8d54a7950aa74862 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:14.122183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1ca1f-e2af-4158-b7d9-76fb9b6bc3eb service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:14.137975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1ca1f-e2af-4158-b7d9-76fb9b6bc3eb service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:14.211521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:14.212460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.212746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.213097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] PATH_INFO: `/v3/users/977937e98ffd4308a39bc605b7f7fe94` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.214319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=977937e98ffd4308a39bc605b7f7fe94)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:14.216661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:14.270239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '977937e98ffd4308a39bc605b7f7fe94'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.270899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '977937e98ffd4308a39bc605b7f7fe94'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.279001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '977937e98ffd4308a39bc605b7f7fe94'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.279659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '977937e98ffd4308a39bc605b7f7fe94'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.291390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-55d35f32-11e9-40d2-ac66-8f8354ca0140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '977937e98ffd4308a39bc605b7f7fe94'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.303521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 990/2005] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:14 2022] DELETE /identity/v3/users/977937e98ffd4308a39bc605b7f7fe94 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:14.470440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:14.472123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:14.491453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6aff8a-a126-460e-a42f-43d2f5aaab9d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:14.549591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:14.551861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.552151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.552436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] PATH_INFO: `/v3/projects/a3217b6fd0f24e23958838338a3e7ea4` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.553684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3217b6fd0f24e23958838338a3e7ea4)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:14.557481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:14.606537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3217b6fd0f24e23958838338a3e7ea4', 'request_id': 'req-5cbdc16a-4ddd-48c6-9609-64de5255977c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.612971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3217b6fd0f24e23958838338a3e7ea4', 'request_id': 'req-5cbdc16a-4ddd-48c6-9609-64de5255977c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.615084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:14.619609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3217b6fd0f24e23958838338a3e7ea4', 'request_id': 'req-5cbdc16a-4ddd-48c6-9609-64de5255977c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.632981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] The token cache is being invalidate because project a3217b6fd0f24e23958838338a3e7ea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:14.633273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.633895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5cbdc16a-4ddd-48c6-9609-64de5255977c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:14.635449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1016/2006] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:14 2022] DELETE /identity/v3/projects/a3217b6fd0f24e23958838338a3e7ea4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:14.641247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b5911-b614-4dbb-846e-cd15d3eae739 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.641537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b5911-b614-4dbb-846e-cd15d3eae739 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.641879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b5911-b614-4dbb-846e-cd15d3eae739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.660157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b82b5911-b614-4dbb-846e-cd15d3eae739 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:14.661420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b82b5911-b614-4dbb-846e-cd15d3eae739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:14.720171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 991/2007] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:14.728014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b1304-40bd-4cf2-891d-6222891b742e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.728417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b1304-40bd-4cf2-891d-6222891b742e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.728579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b1304-40bd-4cf2-891d-6222891b742e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.747840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-be7b1304-40bd-4cf2-891d-6222891b742e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:14.748620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-be7b1304-40bd-4cf2-891d-6222891b742e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:14.756669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1017/2008] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:14.760849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b5911-b614-4dbb-846e-cd15d3eae739 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:14.772002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b5911-b614-4dbb-846e-cd15d3eae739 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:14.776570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a01b0a-c69a-451c-becf-da5cf178fedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:14.777377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01b0a-c69a-451c-becf-da5cf178fedb admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.777654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01b0a-c69a-451c-becf-da5cf178fedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.777926 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01b0a-c69a-451c-becf-da5cf178fedb admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.778653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a01b0a-c69a-451c-becf-da5cf178fedb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:14.778959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a01b0a-c69a-451c-becf-da5cf178fedb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:14.780698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a01b0a-c69a-451c-becf-da5cf178fedb admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:14.787630 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:14.795955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 992/2009] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:14.798042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b1304-40bd-4cf2-891d-6222891b742e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:14.807466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b1304-40bd-4cf2-891d-6222891b742e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:14.812624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad8490-f79d-4461-919f-eccf5ea98224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:14.813299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad8490-f79d-4461-919f-eccf5ea98224 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.813572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad8490-f79d-4461-919f-eccf5ea98224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.813848 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad8490-f79d-4461-919f-eccf5ea98224 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.814546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad8490-f79d-4461-919f-eccf5ea98224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:14.816436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad8490-f79d-4461-919f-eccf5ea98224 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:14.844718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1018/2010] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:14 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:14.849376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a01b0a-c69a-451c-becf-da5cf178fedb admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:14.860258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a01b0a-c69a-451c-becf-da5cf178fedb admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:14.918697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25500be-8a3d-42fb-adf6-ff6e059bbb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:14.919373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25500be-8a3d-42fb-adf6-ff6e059bbb90 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.919657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25500be-8a3d-42fb-adf6-ff6e059bbb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.919934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25500be-8a3d-42fb-adf6-ff6e059bbb90 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.920615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25500be-8a3d-42fb-adf6-ff6e059bbb90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:14.922598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25500be-8a3d-42fb-adf6-ff6e059bbb90 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:14.942529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 993/2011] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:14 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:14.946671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad8490-f79d-4461-919f-eccf5ea98224 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:14.957546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad8490-f79d-4461-919f-eccf5ea98224 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:14.962168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a8479-ed22-4c22-a2eb-060f175a0ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:14.962847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a8479-ed22-4c22-a2eb-060f175a0ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.963131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a8479-ed22-4c22-a2eb-060f175a0ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.963411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a8479-ed22-4c22-a2eb-060f175a0ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.964156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a8479-ed22-4c22-a2eb-060f175a0ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:14.966427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a8479-ed22-4c22-a2eb-060f175a0ab2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:14.978012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1019/2012] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:14.981798 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25500be-8a3d-42fb-adf6-ff6e059bbb90 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:14.992579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25500be-8a3d-42fb-adf6-ff6e059bbb90 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:14.997388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46edbab-8dda-41c2-81a2-fcf5e3d40433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:14.998054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46edbab-8dda-41c2-81a2-fcf5e3d40433 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:14.998368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46edbab-8dda-41c2-81a2-fcf5e3d40433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:14.998648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46edbab-8dda-41c2-81a2-fcf5e3d40433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:14.999374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46edbab-8dda-41c2-81a2-fcf5e3d40433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:15.001016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46edbab-8dda-41c2-81a2-fcf5e3d40433 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:15.012313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 994/2013] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:15.016256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a8479-ed22-4c22-a2eb-060f175a0ab2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:15.026511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a8479-ed22-4c22-a2eb-060f175a0ab2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:15.031769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a033c2-8aba-4a32-82d5-a7cc3dd0444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:15.032644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a033c2-8aba-4a32-82d5-a7cc3dd0444d admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:15.032951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a033c2-8aba-4a32-82d5-a7cc3dd0444d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:15.033226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a033c2-8aba-4a32-82d5-a7cc3dd0444d admin admin] PATH_INFO: `/v3/projects/a6efdb7fad944df394cfaeca41e39afd/users/d8559820a2f94cd7a557ed4c7bb93d2d/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:15.047901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a033c2-8aba-4a32-82d5-a7cc3dd0444d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6efdb7fad944df394cfaeca41e39afd, user_id=d8559820a2f94cd7a557ed4c7bb93d2d, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:15.050664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a033c2-8aba-4a32-82d5-a7cc3dd0444d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:15.063770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1020/2014] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:15 2022] PUT /identity/v3/projects/a6efdb7fad944df394cfaeca41e39afd/users/d8559820a2f94cd7a557ed4c7bb93d2d/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:15.075584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd16e7-f71f-4f50-b884-3a6398b58d39 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:15.075908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd16e7-f71f-4f50-b884-3a6398b58d39 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:15.076222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd16e7-f71f-4f50-b884-3a6398b58d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:15.103903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a7bd16e7-f71f-4f50-b884-3a6398b58d39 None None] MFA Rules not processed for user `d8559820a2f94cd7a557ed4c7bb93d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:15.105149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a7bd16e7-f71f-4f50-b884-3a6398b58d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:15.178227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 995/2015] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:15.189823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a033c2-8aba-4a32-82d5-a7cc3dd0444d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:15.192176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b7a033c2-8aba-4a32-82d5-a7cc3dd0444d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:15.256758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a033c2-8aba-4a32-82d5-a7cc3dd0444d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:15.260789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc75bd-dead-4669-82c2-7dbd0e2d8085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:15.261860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc75bd-dead-4669-82c2-7dbd0e2d8085 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:15.262139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc75bd-dead-4669-82c2-7dbd0e2d8085 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:15.262556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc75bd-dead-4669-82c2-7dbd0e2d8085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:15.271391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc75bd-dead-4669-82c2-7dbd0e2d8085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:15.272973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc75bd-dead-4669-82c2-7dbd0e2d8085 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:15.282840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1021/2016] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:15.302595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd16e7-f71f-4f50-b884-3a6398b58d39 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:15.365851 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd16e7-f71f-4f50-b884-3a6398b58d39 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:15.370422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:15.371490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:15.371771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:15.372043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] PATH_INFO: `/v3/users/d8559820a2f94cd7a557ed4c7bb93d2d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:15.373320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8559820a2f94cd7a557ed4c7bb93d2d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:15.375359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:15.380695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:15.421294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8559820a2f94cd7a557ed4c7bb93d2d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:15.422004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8559820a2f94cd7a557ed4c7bb93d2d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:15.429051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8559820a2f94cd7a557ed4c7bb93d2d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:15.429693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8559820a2f94cd7a557ed4c7bb93d2d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:15.441728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8559820a2f94cd7a557ed4c7bb93d2d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:15.454236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 996/2017] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:15 2022] DELETE /identity/v3/users/d8559820a2f94cd7a557ed4c7bb93d2d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:15.458695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff134b80-288a-45cf-b005-f7e46ab03462 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:15.458987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff134b80-288a-45cf-b005-f7e46ab03462 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:15.459283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff134b80-288a-45cf-b005-f7e46ab03462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:15.476572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ff134b80-288a-45cf-b005-f7e46ab03462 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:15.477800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ff134b80-288a-45cf-b005-f7e46ab03462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:15.537332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1022/2018] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:15.547509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:15.548955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:15.606934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511655c0-dc45-42e5-812b-0eb0361c68c8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:15.610791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583b268-2569-42f0-8966-c813b55fc6ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:15.611274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583b268-2569-42f0-8966-c813b55fc6ba service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:15.611558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583b268-2569-42f0-8966-c813b55fc6ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:15.611835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583b268-2569-42f0-8966-c813b55fc6ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:15.621567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583b268-2569-42f0-8966-c813b55fc6ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:15.623018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583b268-2569-42f0-8966-c813b55fc6ba service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:15.632439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 997/2019] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:16.106267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff134b80-288a-45cf-b005-f7e46ab03462 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:16.107772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ff134b80-288a-45cf-b005-f7e46ab03462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:16.120007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff134b80-288a-45cf-b005-f7e46ab03462 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:16.124822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:16.125722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.126005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.126282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] PATH_INFO: `/v3/projects/a6efdb7fad944df394cfaeca41e39afd` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.127461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6efdb7fad944df394cfaeca41e39afd)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:16.129410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:16.151010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:16.179729 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6efdb7fad944df394cfaeca41e39afd', 'request_id': 'req-7c8bc0b6-f64c-4202-9dad-b999286e9b34'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:16.186326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6efdb7fad944df394cfaeca41e39afd', 'request_id': 'req-7c8bc0b6-f64c-4202-9dad-b999286e9b34'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:16.192175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6efdb7fad944df394cfaeca41e39afd', 'request_id': 'req-7c8bc0b6-f64c-4202-9dad-b999286e9b34'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:16.205575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] The token cache is being invalidate because project a6efdb7fad944df394cfaeca41e39afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:16.205865 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:16.206522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7c8bc0b6-f64c-4202-9dad-b999286e9b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:16.208075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1023/2020] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:16 2022] DELETE /identity/v3/projects/a6efdb7fad944df394cfaeca41e39afd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:16.213007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cf50c-4e97-4781-b3fd-b5b8ac0c81f5 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.213470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cf50c-4e97-4781-b3fd-b5b8ac0c81f5 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.213562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cf50c-4e97-4781-b3fd-b5b8ac0c81f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.232688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d18cf50c-4e97-4781-b3fd-b5b8ac0c81f5 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:16.233936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d18cf50c-4e97-4781-b3fd-b5b8ac0c81f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:16.295351 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 998/2021] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:16.303197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8029f83-fcfa-419e-aac7-58f18082bfce None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.303429 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8029f83-fcfa-419e-aac7-58f18082bfce None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.303706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8029f83-fcfa-419e-aac7-58f18082bfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.323042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d8029f83-fcfa-419e-aac7-58f18082bfce None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:16.325061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d8029f83-fcfa-419e-aac7-58f18082bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:16.332889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1024/2022] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:16.336996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cf50c-4e97-4781-b3fd-b5b8ac0c81f5 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:16.346986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cf50c-4e97-4781-b3fd-b5b8ac0c81f5 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:16.351575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb4922-f68d-40f0-a487-1e16a09bf2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:16.352229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb4922-f68d-40f0-a487-1e16a09bf2da admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.352508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb4922-f68d-40f0-a487-1e16a09bf2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.352777 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb4922-f68d-40f0-a487-1e16a09bf2da admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.353771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb4922-f68d-40f0-a487-1e16a09bf2da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:16.353890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb4922-f68d-40f0-a487-1e16a09bf2da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:16.355919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb4922-f68d-40f0-a487-1e16a09bf2da admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:16.368185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 999/2023] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:31:16.373470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8029f83-fcfa-419e-aac7-58f18082bfce None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:16.383449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8029f83-fcfa-419e-aac7-58f18082bfce None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:16.388861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c168ee-935f-486c-84c2-ab8163810286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:16.389558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c168ee-935f-486c-84c2-ab8163810286 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.389843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c168ee-935f-486c-84c2-ab8163810286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.390125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c168ee-935f-486c-84c2-ab8163810286 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.390832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c168ee-935f-486c-84c2-ab8163810286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:16.392764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c168ee-935f-486c-84c2-ab8163810286 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:16.423125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1025/2024] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:16 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:16.424664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb4922-f68d-40f0-a487-1e16a09bf2da admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:16.435475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb4922-f68d-40f0-a487-1e16a09bf2da admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:16.447277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:16.502624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d21a5-d11f-4d6d-af94-073a3396fd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:16.503300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d21a5-d11f-4d6d-af94-073a3396fd78 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.503584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d21a5-d11f-4d6d-af94-073a3396fd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.503860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d21a5-d11f-4d6d-af94-073a3396fd78 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.504583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d21a5-d11f-4d6d-af94-073a3396fd78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:16.506585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d21a5-d11f-4d6d-af94-073a3396fd78 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:16.526861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1000/2025] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:16 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:16.530902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c168ee-935f-486c-84c2-ab8163810286 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:16.541213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c168ee-935f-486c-84c2-ab8163810286 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:16.546435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea878e5-3c49-40a1-8baa-d7f7933f65b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:16.547098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea878e5-3c49-40a1-8baa-d7f7933f65b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.547420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea878e5-3c49-40a1-8baa-d7f7933f65b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.547677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea878e5-3c49-40a1-8baa-d7f7933f65b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.548396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea878e5-3c49-40a1-8baa-d7f7933f65b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:16.550014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea878e5-3c49-40a1-8baa-d7f7933f65b2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:16.561442 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1026/2026] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:16.565396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d21a5-d11f-4d6d-af94-073a3396fd78 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:16.575208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d21a5-d11f-4d6d-af94-073a3396fd78 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:16.579826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90fc17-0972-4c31-9325-fbc4340a4189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:16.580479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90fc17-0972-4c31-9325-fbc4340a4189 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.580763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90fc17-0972-4c31-9325-fbc4340a4189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.581076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90fc17-0972-4c31-9325-fbc4340a4189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.581807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90fc17-0972-4c31-9325-fbc4340a4189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:16.583397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90fc17-0972-4c31-9325-fbc4340a4189 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:16.594247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1001/2027] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:16.598638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea878e5-3c49-40a1-8baa-d7f7933f65b2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:16.610924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea878e5-3c49-40a1-8baa-d7f7933f65b2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:16.615248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d8530-fb0f-4865-9d82-6644268301db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:16.616136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8530-fb0f-4865-9d82-6644268301db admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.616420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8530-fb0f-4865-9d82-6644268301db admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.616701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8530-fb0f-4865-9d82-6644268301db admin admin] PATH_INFO: `/v3/projects/2f9d20f74acb4df68802f42512f81b02/users/8d5be27c60a24e8083c99b1d1160c7b2/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.631896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d8530-fb0f-4865-9d82-6644268301db admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9d20f74acb4df68802f42512f81b02, user_id=8d5be27c60a24e8083c99b1d1160c7b2, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:16.634614 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d8530-fb0f-4865-9d82-6644268301db admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:16.648523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1027/2028] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:16 2022] PUT /identity/v3/projects/2f9d20f74acb4df68802f42512f81b02/users/8d5be27c60a24e8083c99b1d1160c7b2/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:16.659670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8ce7-bae1-46dc-a217-d9f5ea5ae4ef None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.659910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8ce7-bae1-46dc-a217-d9f5ea5ae4ef None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.660203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8ce7-bae1-46dc-a217-d9f5ea5ae4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.686662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-1fca8ce7-bae1-46dc-a217-d9f5ea5ae4ef None None] MFA Rules not processed for user `8d5be27c60a24e8083c99b1d1160c7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:16.687955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1fca8ce7-bae1-46dc-a217-d9f5ea5ae4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:16.753428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1002/2029] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:16.764993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d8530-fb0f-4865-9d82-6644268301db admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:16.766679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b59d8530-fb0f-4865-9d82-6644268301db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:16.829138 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d8530-fb0f-4865-9d82-6644268301db admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:16.832853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40221521-e978-49c1-a30e-5521181ffe10 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb0e53d4b3774959820ac72824b3d4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:16.833605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40221521-e978-49c1-a30e-5521181ffe10 service glance] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:16.833873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40221521-e978-49c1-a30e-5521181ffe10 service glance] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:16.834172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40221521-e978-49c1-a30e-5521181ffe10 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:16.844033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40221521-e978-49c1-a30e-5521181ffe10 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:16.845540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40221521-e978-49c1-a30e-5521181ffe10 service glance] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:16.854709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1028/2030] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:19.255327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca8ce7-bae1-46dc-a217-d9f5ea5ae4ef None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:19.256825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1fca8ce7-bae1-46dc-a217-d9f5ea5ae4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:19.325068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca8ce7-bae1-46dc-a217-d9f5ea5ae4ef None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:19.330461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:19.331635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:19.331985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:19.332286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] PATH_INFO: `/v3/users/a6c6dd9e7b304057a076420e7c09df90` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:19.333747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6c6dd9e7b304057a076420e7c09df90)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:19.335957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:19.365345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40221521-e978-49c1-a30e-5521181ffe10 service glance] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:19.367322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-40221521-e978-49c1-a30e-5521181ffe10 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:19.367991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:19.385159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40221521-e978-49c1-a30e-5521181ffe10 service glance] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:19.390977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:19.397394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:19.397394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:19.397394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] PATH_INFO: `/v3/users/e76707e47de34c19a093b5c5d145d17a` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:19.397905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6c6dd9e7b304057a076420e7c09df90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.398006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e76707e47de34c19a093b5c5d145d17a)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:19.398124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6c6dd9e7b304057a076420e7c09df90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.398211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:19.408613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6c6dd9e7b304057a076420e7c09df90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.409444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6c6dd9e7b304057a076420e7c09df90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.437681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e95d2696-608c-4ad4-914a-e1a68a7ce558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6c6dd9e7b304057a076420e7c09df90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.456790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:19.461165 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1003/2031] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:19 2022] DELETE /identity/v3/users/a6c6dd9e7b304057a076420e7c09df90 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:19.463191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706eb5-4e39-4e60-b061-0aa761ff7241 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:19.463491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706eb5-4e39-4e60-b061-0aa761ff7241 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:19.463769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706eb5-4e39-4e60-b061-0aa761ff7241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:19.477099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e76707e47de34c19a093b5c5d145d17a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.494147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a7706eb5-4e39-4e60-b061-0aa761ff7241 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:19.494457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e76707e47de34c19a093b5c5d145d17a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.496560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a7706eb5-4e39-4e60-b061-0aa761ff7241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:19.503487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e76707e47de34c19a093b5c5d145d17a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.504399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e76707e47de34c19a093b5c5d145d17a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.505448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e2a092a7-6558-464c-b16b-0160e6658fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e76707e47de34c19a093b5c5d145d17a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:19.530738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1029/2032] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:19 2022] DELETE /identity/v3/users/e76707e47de34c19a093b5c5d145d17a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:19.535614 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2d433-57a3-44d1-9b64-a2c5cbc18190 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:19.535977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2d433-57a3-44d1-9b64-a2c5cbc18190 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:19.537173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2d433-57a3-44d1-9b64-a2c5cbc18190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:19.564183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-47a2d433-57a3-44d1-9b64-a2c5cbc18190 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:19.565716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-47a2d433-57a3-44d1-9b64-a2c5cbc18190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:19.567201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1004/2033] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:19.578180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7706eb5-4e39-4e60-b061-0aa761ff7241 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:19.579708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a7706eb5-4e39-4e60-b061-0aa761ff7241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:19.647842 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1030/2034] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:19.657709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7706eb5-4e39-4e60-b061-0aa761ff7241 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:19.658308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2d433-57a3-44d1-9b64-a2c5cbc18190 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:19.659777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-47a2d433-57a3-44d1-9b64-a2c5cbc18190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:19.710381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de85027-f286-4898-8ce2-175f44201ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:19.711141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de85027-f286-4898-8ce2-175f44201ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:19.711424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de85027-f286-4898-8ce2-175f44201ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:19.711704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de85027-f286-4898-8ce2-175f44201ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:19.720220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2d433-57a3-44d1-9b64-a2c5cbc18190 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:19.722230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de85027-f286-4898-8ce2-175f44201ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:19.723728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de85027-f286-4898-8ce2-175f44201ea8 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:19.723925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f44d1-6dea-49c2-9f09-3a51aae43454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:19.724573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f44d1-6dea-49c2-9f09-3a51aae43454 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:19.724843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f44d1-6dea-49c2-9f09-3a51aae43454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:19.725144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f44d1-6dea-49c2-9f09-3a51aae43454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:19.734384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1005/2035] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:31:19.735149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f44d1-6dea-49c2-9f09-3a51aae43454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:19.736663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f44d1-6dea-49c2-9f09-3a51aae43454 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:19.746712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1031/2036] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:31:20.122867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de85027-f286-4898-8ce2-175f44201ea8 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.124336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3de85027-f286-4898-8ce2-175f44201ea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:20.133353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f44d1-6dea-49c2-9f09-3a51aae43454 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.135283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:20.138786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-733f44d1-6dea-49c2-9f09-3a51aae43454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:20.155436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de85027-f286-4898-8ce2-175f44201ea8 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.162238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.163182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.163465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.163744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] PATH_INFO: `/v3/projects/5bc7e3b869ea4ac39e61c50f86e2cd6b` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.165310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc7e3b869ea4ac39e61c50f86e2cd6b)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.166989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f44d1-6dea-49c2-9f09-3a51aae43454 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.167484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.175727 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.177488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.177755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.178036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] PATH_INFO: `/v3/projects/934e97feb41542908e28114363996e77` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.179414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=934e97feb41542908e28114363996e77)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.181402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.220650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc7e3b869ea4ac39e61c50f86e2cd6b', 'request_id': 'req-be9a619b-0012-451d-bfd0-f51e097b93d9'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.226510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc7e3b869ea4ac39e61c50f86e2cd6b', 'request_id': 'req-be9a619b-0012-451d-bfd0-f51e097b93d9'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.234912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc7e3b869ea4ac39e61c50f86e2cd6b', 'request_id': 'req-be9a619b-0012-451d-bfd0-f51e097b93d9'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.237358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934e97feb41542908e28114363996e77', 'request_id': 'req-48badfa0-b5e6-45ae-a437-b36ccb183861'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.244130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934e97feb41542908e28114363996e77', 'request_id': 'req-48badfa0-b5e6-45ae-a437-b36ccb183861'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.247904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:20.252375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934e97feb41542908e28114363996e77', 'request_id': 'req-48badfa0-b5e6-45ae-a437-b36ccb183861'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.253829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] The token cache is being invalidate because project 5bc7e3b869ea4ac39e61c50f86e2cd6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:20.254145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.255316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-be9a619b-0012-451d-bfd0-f51e097b93d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.257890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1006/2037] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:20 2022] DELETE /identity/v3/projects/5bc7e3b869ea4ac39e61c50f86e2cd6b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:20.264381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ebf29-d680-45c2-a4b3-bca3a427b415 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.264676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ebf29-d680-45c2-a4b3-bca3a427b415 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.264989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ebf29-d680-45c2-a4b3-bca3a427b415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.268260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] The token cache is being invalidate because project 934e97feb41542908e28114363996e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:20.268547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.269217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.270734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1032/2038] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:20 2022] DELETE /identity/v3/projects/934e97feb41542908e28114363996e77 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:20.274522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.275802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:20.284392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-0a3ebf29-d680-45c2-a4b3-bca3a427b415 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:20.285610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0a3ebf29-d680-45c2-a4b3-bca3a427b415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:20.352117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1007/2039] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:20.352385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48badfa0-b5e6-45ae-a437-b36ccb183861 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.357853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.358735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.359018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.359296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] PATH_INFO: `/v3/users/fdf25b7a8bc343b0a15e4c9be71df4be` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.360504 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdf25b7a8bc343b0a15e4c9be71df4be)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.361244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1f92e-4181-40cc-91ef-47f69daef555 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.361549 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1f92e-4181-40cc-91ef-47f69daef555 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.361821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1f92e-4181-40cc-91ef-47f69daef555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.362475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.381755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-fbc1f92e-4181-40cc-91ef-47f69daef555 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:20.382891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-fbc1f92e-4181-40cc-91ef-47f69daef555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:20.392198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1008/2040] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:20.396858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1f92e-4181-40cc-91ef-47f69daef555 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.406670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1f92e-4181-40cc-91ef-47f69daef555 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.411645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120d283-6678-4eac-812e-7897a25b882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.412291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120d283-6678-4eac-812e-7897a25b882d admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.412399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdf25b7a8bc343b0a15e4c9be71df4be'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.412960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120d283-6678-4eac-812e-7897a25b882d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.413057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120d283-6678-4eac-812e-7897a25b882d admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.413732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120d283-6678-4eac-812e-7897a25b882d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:20.414055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120d283-6678-4eac-812e-7897a25b882d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.415747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120d283-6678-4eac-812e-7897a25b882d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.429849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdf25b7a8bc343b0a15e4c9be71df4be'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.431101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1009/2041] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:20.435628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120d283-6678-4eac-812e-7897a25b882d admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.440495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdf25b7a8bc343b0a15e4c9be71df4be'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.441355 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdf25b7a8bc343b0a15e4c9be71df4be'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.442083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdf25b7a8bc343b0a15e4c9be71df4be'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.446928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120d283-6678-4eac-812e-7897a25b882d admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.452749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae46bf6-00ec-48e0-a386-2d734ff66dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.453450 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae46bf6-00ec-48e0-a386-2d734ff66dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.453738 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae46bf6-00ec-48e0-a386-2d734ff66dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.454012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae46bf6-00ec-48e0-a386-2d734ff66dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.454736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae46bf6-00ec-48e0-a386-2d734ff66dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.456651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae46bf6-00ec-48e0-a386-2d734ff66dc8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.457144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1033/2042] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:20 2022] DELETE /identity/v3/users/fdf25b7a8bc343b0a15e4c9be71df4be => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:20.489165 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1010/2043] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:20 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:31:20.493717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.495211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:20.576910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb5aaf-a293-4943-979b-aad7b1e18dcd admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.594145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:20.665693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f72de-15b1-4696-b633-fdfc689cf2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.666565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f72de-15b1-4696-b633-fdfc689cf2de admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.667015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f72de-15b1-4696-b633-fdfc689cf2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.667448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f72de-15b1-4696-b633-fdfc689cf2de admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.669260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f72de-15b1-4696-b633-fdfc689cf2de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.671413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f72de-15b1-4696-b633-fdfc689cf2de admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.707313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1034/2044] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:20 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:20.711446 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae46bf6-00ec-48e0-a386-2d734ff66dc8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.724109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae46bf6-00ec-48e0-a386-2d734ff66dc8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.732306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290b860-6ffd-45fd-b777-1f344816b120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.733018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290b860-6ffd-45fd-b777-1f344816b120 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.733332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290b860-6ffd-45fd-b777-1f344816b120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.733625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290b860-6ffd-45fd-b777-1f344816b120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.734365 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290b860-6ffd-45fd-b777-1f344816b120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.736145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290b860-6ffd-45fd-b777-1f344816b120 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.750240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1011/2045] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:20.755029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f72de-15b1-4696-b633-fdfc689cf2de admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.767753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f72de-15b1-4696-b633-fdfc689cf2de admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.772831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9f716-95e7-4730-a53e-d38b11c08a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.774398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9f716-95e7-4730-a53e-d38b11c08a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.774666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9f716-95e7-4730-a53e-d38b11c08a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.774938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9f716-95e7-4730-a53e-d38b11c08a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.775806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9f716-95e7-4730-a53e-d38b11c08a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.778433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9f716-95e7-4730-a53e-d38b11c08a75 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.800261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1035/2046] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:20.801644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290b860-6ffd-45fd-b777-1f344816b120 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.802045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9f716-95e7-4730-a53e-d38b11c08a75 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.805326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8290b860-6ffd-45fd-b777-1f344816b120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:20.823423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290b860-6ffd-45fd-b777-1f344816b120 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.823963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9f716-95e7-4730-a53e-d38b11c08a75 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.832069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.832998 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.833270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe59e1-788b-4aef-8d47-650b9f66c586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:20.833405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.833550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] PATH_INFO: `/v3/projects/d13bf39a43624a6d9dfa2d8232138307` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.834179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe59e1-788b-4aef-8d47-650b9f66c586 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.834465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe59e1-788b-4aef-8d47-650b9f66c586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.834765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe59e1-788b-4aef-8d47-650b9f66c586 admin admin] PATH_INFO: `/v3/projects/591802cf5c044932940307866f388790/users/ceed468da0c844b688f5f91768b0dd96/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.835024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d13bf39a43624a6d9dfa2d8232138307)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.837066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.851562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe59e1-788b-4aef-8d47-650b9f66c586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591802cf5c044932940307866f388790, user_id=ceed468da0c844b688f5f91768b0dd96, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:20.854285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe59e1-788b-4aef-8d47-650b9f66c586 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:20.868147 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1036/2047] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:20 2022] PUT /identity/v3/projects/591802cf5c044932940307866f388790/users/ceed468da0c844b688f5f91768b0dd96/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:20.869223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe59e1-788b-4aef-8d47-650b9f66c586 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:20.870883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1afe59e1-788b-4aef-8d47-650b9f66c586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:20.877407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:20.897999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd13bf39a43624a6d9dfa2d8232138307', 'request_id': 'req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.904690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd13bf39a43624a6d9dfa2d8232138307', 'request_id': 'req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.912339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd13bf39a43624a6d9dfa2d8232138307', 'request_id': 'req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.925605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] The token cache is being invalidate because project d13bf39a43624a6d9dfa2d8232138307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:20.925898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.926561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d58d22a4-cf39-4d72-8cda-d54e9af1ff2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:20.928216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1012/2048] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:20 2022] DELETE /identity/v3/projects/d13bf39a43624a6d9dfa2d8232138307 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:20.929577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa9fe8-8ca2-4b1f-871a-201d9ecc2658 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:20.929864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa9fe8-8ca2-4b1f-871a-201d9ecc2658 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:20.930143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa9fe8-8ca2-4b1f-871a-201d9ecc2658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:20.946685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe59e1-788b-4aef-8d47-650b9f66c586 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:20.947721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-3caa9fe8-8ca2-4b1f-871a-201d9ecc2658 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:20.949147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3caa9fe8-8ca2-4b1f-871a-201d9ecc2658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:21.007009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.008106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.008395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.008700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1013/2049] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:21.010055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629995a-aab6-4596-8586-171fecef37cd None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.010343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629995a-aab6-4596-8586-171fecef37cd None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.010627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629995a-aab6-4596-8586-171fecef37cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.013662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] PATH_INFO: `/v3/users/86266900bbfe449aaf6c78027859644b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.015149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=86266900bbfe449aaf6c78027859644b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.017144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.032174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:21.035190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e629995a-aab6-4596-8586-171fecef37cd None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:21.036598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e629995a-aab6-4596-8586-171fecef37cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:21.046498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1014/2050] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:21.047544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629995a-aab6-4596-8586-171fecef37cd None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.048842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e629995a-aab6-4596-8586-171fecef37cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:21.083917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86266900bbfe449aaf6c78027859644b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:21.098551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86266900bbfe449aaf6c78027859644b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:21.106467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86266900bbfe449aaf6c78027859644b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:21.107409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86266900bbfe449aaf6c78027859644b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:21.108194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-17f24ea2-f089-42ef-9512-967ad1b07a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86266900bbfe449aaf6c78027859644b'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:21.118190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629995a-aab6-4596-8586-171fecef37cd None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.122087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1037/2051] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:20 2022] DELETE /identity/v3/users/86266900bbfe449aaf6c78027859644b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:21.122240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f00311-e595-42db-90f3-c01b3e4acebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.122557 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f00311-e595-42db-90f3-c01b3e4acebf service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.122838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f00311-e595-42db-90f3-c01b3e4acebf service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.123114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f00311-e595-42db-90f3-c01b3e4acebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.125008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-78f00311-e595-42db-90f3-c01b3e4acebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:21.126480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51afbff-997d-4f5c-a8f7-d7a3381c0720 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.126764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51afbff-997d-4f5c-a8f7-d7a3381c0720 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.127038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51afbff-997d-4f5c-a8f7-d7a3381c0720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.138487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f00311-e595-42db-90f3-c01b3e4acebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.139945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f00311-e595-42db-90f3-c01b3e4acebf service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.150997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1015/2052] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:21.152247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748e0ec-a4b1-4f98-af85-c3135f820183 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.153069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748e0ec-a4b1-4f98-af85-c3135f820183 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.153348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748e0ec-a4b1-4f98-af85-c3135f820183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.153460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a51afbff-997d-4f5c-a8f7-d7a3381c0720 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:21.154611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a51afbff-997d-4f5c-a8f7-d7a3381c0720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:21.171549 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b748e0ec-a4b1-4f98-af85-c3135f820183 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:21.173227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b748e0ec-a4b1-4f98-af85-c3135f820183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:21.223631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1038/2053] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:21.228323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51afbff-997d-4f5c-a8f7-d7a3381c0720 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.237855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:21.240891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51afbff-997d-4f5c-a8f7-d7a3381c0720 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.253121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1016/2054] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:21.258423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af3da0-c005-4643-9e24-9ce47bdbc33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.259129 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af3da0-c005-4643-9e24-9ce47bdbc33c admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.259413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af3da0-c005-4643-9e24-9ce47bdbc33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.260154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af3da0-c005-4643-9e24-9ce47bdbc33c admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.261255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af3da0-c005-4643-9e24-9ce47bdbc33c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:21.262078 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af3da0-c005-4643-9e24-9ce47bdbc33c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.264734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af3da0-c005-4643-9e24-9ce47bdbc33c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.273279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748e0ec-a4b1-4f98-af85-c3135f820183 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.273279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b748e0ec-a4b1-4f98-af85-c3135f820183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:21.284149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1039/2055] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:21.289747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af3da0-c005-4643-9e24-9ce47bdbc33c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.305068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af3da0-c005-4643-9e24-9ce47bdbc33c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.311458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93f529-f3ca-4016-a915-2560d0c71038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.312134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93f529-f3ca-4016-a915-2560d0c71038 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.312508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93f529-f3ca-4016-a915-2560d0c71038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.312776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93f529-f3ca-4016-a915-2560d0c71038 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.313447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93f529-f3ca-4016-a915-2560d0c71038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.315394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93f529-f3ca-4016-a915-2560d0c71038 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.347728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748e0ec-a4b1-4f98-af85-c3135f820183 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.351611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1040/2056] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:21 2022] POST /identity/v3/projects => generated 409 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:21.354049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93f529-f3ca-4016-a915-2560d0c71038 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.415143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b54d9-f150-4ae3-9110-17fc69fdf1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.416048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b54d9-f150-4ae3-9110-17fc69fdf1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.416340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b54d9-f150-4ae3-9110-17fc69fdf1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.416621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b54d9-f150-4ae3-9110-17fc69fdf1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.436333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:21.440736 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93f529-f3ca-4016-a915-2560d0c71038 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.448643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457c82e-203a-45e9-b93c-6a5a092267e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.449660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457c82e-203a-45e9-b93c-6a5a092267e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.450185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457c82e-203a-45e9-b93c-6a5a092267e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.451652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457c82e-203a-45e9-b93c-6a5a092267e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.452180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457c82e-203a-45e9-b93c-6a5a092267e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.460791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457c82e-203a-45e9-b93c-6a5a092267e8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.500545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b54d9-f150-4ae3-9110-17fc69fdf1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.501227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1041/2057] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:21 2022] POST /identity/v3/users => generated 305 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:21.504603 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b54d9-f150-4ae3-9110-17fc69fdf1a9 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.506144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457c82e-203a-45e9-b93c-6a5a092267e8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.517270 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1017/2058] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:31:21.518522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457c82e-203a-45e9-b93c-6a5a092267e8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.523328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713bc97-012d-49ea-b186-6cbabf7de256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.523997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713bc97-012d-49ea-b186-6cbabf7de256 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.524279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713bc97-012d-49ea-b186-6cbabf7de256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.524554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713bc97-012d-49ea-b186-6cbabf7de256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.525414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713bc97-012d-49ea-b186-6cbabf7de256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.527406 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713bc97-012d-49ea-b186-6cbabf7de256 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.548548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1042/2059] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:21.549970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b54d9-f150-4ae3-9110-17fc69fdf1a9 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.569116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b54d9-f150-4ae3-9110-17fc69fdf1a9 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.579336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c4fc3-9227-4e76-b64d-909b03a5a35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.580202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c4fc3-9227-4e76-b64d-909b03a5a35d admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.580562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c4fc3-9227-4e76-b64d-909b03a5a35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.580944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c4fc3-9227-4e76-b64d-909b03a5a35d admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.582166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c4fc3-9227-4e76-b64d-909b03a5a35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.584761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c4fc3-9227-4e76-b64d-909b03a5a35d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.616910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1018/2060] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:21.620534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713bc97-012d-49ea-b186-6cbabf7de256 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.631785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713bc97-012d-49ea-b186-6cbabf7de256 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.637323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674e815-0566-4552-8008-8d4f2c46f8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.638293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674e815-0566-4552-8008-8d4f2c46f8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.638673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674e815-0566-4552-8008-8d4f2c46f8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.639028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674e815-0566-4552-8008-8d4f2c46f8a1 admin admin] PATH_INFO: `/v3/projects/f161514ca06b41e3898674a8be756209/users/477dc35a7cc547508b9bafebc7581ad8/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.655419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674e815-0566-4552-8008-8d4f2c46f8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f161514ca06b41e3898674a8be756209, user_id=477dc35a7cc547508b9bafebc7581ad8, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.658601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674e815-0566-4552-8008-8d4f2c46f8a1 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.679418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1043/2061] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:21 2022] PUT /identity/v3/projects/f161514ca06b41e3898674a8be756209/users/477dc35a7cc547508b9bafebc7581ad8/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:21.691575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26a47d-8840-4e75-8142-2eea13fb6ded None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.691867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26a47d-8840-4e75-8142-2eea13fb6ded None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.692145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26a47d-8840-4e75-8142-2eea13fb6ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.725416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-7d26a47d-8840-4e75-8142-2eea13fb6ded None None] MFA Rules not processed for user `477dc35a7cc547508b9bafebc7581ad8`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:21.726813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7d26a47d-8840-4e75-8142-2eea13fb6ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:21.771228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:21.800951 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1019/2062] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:21.805924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674e815-0566-4552-8008-8d4f2c46f8a1 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.896899 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674e815-0566-4552-8008-8d4f2c46f8a1 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.904011 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abf1a9-a093-4e25-a072-42b1393b4811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:21.905116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abf1a9-a093-4e25-a072-42b1393b4811 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:21.905636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abf1a9-a093-4e25-a072-42b1393b4811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:21.906097 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abf1a9-a093-4e25-a072-42b1393b4811 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:21.906951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abf1a9-a093-4e25-a072-42b1393b4811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:21.909349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abf1a9-a093-4e25-a072-42b1393b4811 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:21.943848 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1044/2063] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:21 2022] POST /identity/v3/projects => generated 409 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:21.947840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26a47d-8840-4e75-8142-2eea13fb6ded None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.959588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26a47d-8840-4e75-8142-2eea13fb6ded None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:21.981486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abf1a9-a093-4e25-a072-42b1393b4811 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:21.983395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f8abf1a9-a093-4e25-a072-42b1393b4811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:22.056888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8766c-8323-4f32-9dbc-c747712c3ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.060907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8766c-8323-4f32-9dbc-c747712c3ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.060907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8766c-8323-4f32-9dbc-c747712c3ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.060907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8766c-8323-4f32-9dbc-c747712c3ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.060907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8766c-8323-4f32-9dbc-c747712c3ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.061959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8766c-8323-4f32-9dbc-c747712c3ed6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.083229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abf1a9-a093-4e25-a072-42b1393b4811 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.089204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.090742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.091130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.091307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] PATH_INFO: `/v3/projects/7a9e26fa74704be18c95c9572714c4f6` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.091797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1020/2064] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:21 2022] POST /identity/v3/users => generated 311 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:22.093365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9e26fa74704be18c95c9572714c4f6)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.095677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.095918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8766c-8323-4f32-9dbc-c747712c3ed6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.111079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8766c-8323-4f32-9dbc-c747712c3ed6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.117594 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c3ad0-7573-4bdb-8704-ff3aaf60c0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.118408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3ad0-7573-4bdb-8704-ff3aaf60c0ba admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.118970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3ad0-7573-4bdb-8704-ff3aaf60c0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.119467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3ad0-7573-4bdb-8704-ff3aaf60c0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.120367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c3ad0-7573-4bdb-8704-ff3aaf60c0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.123872 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c3ad0-7573-4bdb-8704-ff3aaf60c0ba admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.138111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1021/2065] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:22.143691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c3ad0-7573-4bdb-8704-ff3aaf60c0ba admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.157414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:22.162992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c3ad0-7573-4bdb-8704-ff3aaf60c0ba admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.175044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9e26fa74704be18c95c9572714c4f6', 'request_id': 'req-74d80524-e729-4948-b774-ed6ce85e5d13'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:22.183730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9e26fa74704be18c95c9572714c4f6', 'request_id': 'req-74d80524-e729-4948-b774-ed6ce85e5d13'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:22.196147 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9e26fa74704be18c95c9572714c4f6', 'request_id': 'req-74d80524-e729-4948-b774-ed6ce85e5d13'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:22.223925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] The token cache is being invalidate because project 7a9e26fa74704be18c95c9572714c4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:22.224937 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:22.225607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-74d80524-e729-4948-b774-ed6ce85e5d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:22.227643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1045/2066] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:21 2022] DELETE /identity/v3/projects/7a9e26fa74704be18c95c9572714c4f6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:22.233971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d1204-2e39-4152-8b5b-8fc133be69b0 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.234267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d1204-2e39-4152-8b5b-8fc133be69b0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.234546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d1204-2e39-4152-8b5b-8fc133be69b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.258378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a79ec0-d488-4ff8-9c71-b0d6bde78801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.259539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a79ec0-d488-4ff8-9c71-b0d6bde78801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.259969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a79ec0-d488-4ff8-9c71-b0d6bde78801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.260348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a79ec0-d488-4ff8-9c71-b0d6bde78801 admin admin] PATH_INFO: `/v3/projects/1c3b41651cff482fa45e954d594ef078/users/61a734cc31a94665a223fc1524dbb165/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.266099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-3c1d1204-2e39-4152-8b5b-8fc133be69b0 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:22.266933 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3c1d1204-2e39-4152-8b5b-8fc133be69b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:22.280551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a79ec0-d488-4ff8-9c71-b0d6bde78801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3b41651cff482fa45e954d594ef078, user_id=61a734cc31a94665a223fc1524dbb165, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.283829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a79ec0-d488-4ff8-9c71-b0d6bde78801 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.303270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1046/2067] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:22.304807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a0a69-37e8-4ac9-8630-713a1b33e913 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.305151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a0a69-37e8-4ac9-8630-713a1b33e913 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.305360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1022/2068] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:22 2022] PUT /identity/v3/projects/1c3b41651cff482fa45e954d594ef078/users/61a734cc31a94665a223fc1524dbb165/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:22.305478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a0a69-37e8-4ac9-8630-713a1b33e913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.310437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a79ec0-d488-4ff8-9c71-b0d6bde78801 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.312487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-29a79ec0-d488-4ff8-9c71-b0d6bde78801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:22.357259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b29a0a69-37e8-4ac9-8630-713a1b33e913 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:22.358870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b29a0a69-37e8-4ac9-8630-713a1b33e913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:22.372274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:22.407212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a79ec0-d488-4ff8-9c71-b0d6bde78801 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.414150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0c817-7984-4da6-88ef-a825489f2e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.415042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0c817-7984-4da6-88ef-a825489f2e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.415665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0c817-7984-4da6-88ef-a825489f2e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.416132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0c817-7984-4da6-88ef-a825489f2e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.417548 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0c817-7984-4da6-88ef-a825489f2e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.420787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0c817-7984-4da6-88ef-a825489f2e4c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.442344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1023/2069] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:22.443682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0c817-7984-4da6-88ef-a825489f2e4c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.459252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0c817-7984-4da6-88ef-a825489f2e4c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.475721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50fbf3-1819-416c-a295-b45af25676dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.476638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50fbf3-1819-416c-a295-b45af25676dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.476959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50fbf3-1819-416c-a295-b45af25676dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.477247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50fbf3-1819-416c-a295-b45af25676dd admin admin] PATH_INFO: `/v3/projects/1c3b41651cff482fa45e954d594ef078/users/61a734cc31a94665a223fc1524dbb165/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.479338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50fbf3-1819-416c-a295-b45af25676dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3b41651cff482fa45e954d594ef078, user_id=61a734cc31a94665a223fc1524dbb165, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.482158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50fbf3-1819-416c-a295-b45af25676dd admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.497493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1047/2070] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:22.506660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a0a69-37e8-4ac9-8630-713a1b33e913 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.514880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a0a69-37e8-4ac9-8630-713a1b33e913 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.516560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1024/2071] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:22 2022] PUT /identity/v3/projects/1c3b41651cff482fa45e954d594ef078/users/61a734cc31a94665a223fc1524dbb165/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:22.527291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dce4c-1767-4a0f-9a2c-80806e261889 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.527686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dce4c-1767-4a0f-9a2c-80806e261889 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.528132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dce4c-1767-4a0f-9a2c-80806e261889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.592106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ee7dce4c-1767-4a0f-9a2c-80806e261889 None None] MFA Rules not processed for user `61a734cc31a94665a223fc1524dbb165`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:22.593396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ee7dce4c-1767-4a0f-9a2c-80806e261889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:22.626664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefc3a1-7610-4907-a615-5e4d242a79ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.627485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc3a1-7610-4907-a615-5e4d242a79ae admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.627778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc3a1-7610-4907-a615-5e4d242a79ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.628055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc3a1-7610-4907-a615-5e4d242a79ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.629017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc3a1-7610-4907-a615-5e4d242a79ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:22.629333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc3a1-7610-4907-a615-5e4d242a79ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.632315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc3a1-7610-4907-a615-5e4d242a79ae admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.649430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1048/2072] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:22.650438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefc3a1-7610-4907-a615-5e4d242a79ae admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.663605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefc3a1-7610-4907-a615-5e4d242a79ae admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.670776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea328776-06c8-4d21-b9f3-5da80504d1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.671463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea328776-06c8-4d21-b9f3-5da80504d1ee admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.671748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea328776-06c8-4d21-b9f3-5da80504d1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.672032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea328776-06c8-4d21-b9f3-5da80504d1ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.672953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea328776-06c8-4d21-b9f3-5da80504d1ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.675061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea328776-06c8-4d21-b9f3-5da80504d1ee admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.708279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1025/2073] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:22.716224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dce4c-1767-4a0f-9a2c-80806e261889 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.718257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ee7dce4c-1767-4a0f-9a2c-80806e261889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:22.722616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1049/2074] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:22 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:22.723703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea328776-06c8-4d21-b9f3-5da80504d1ee admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.786324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:22.807585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dce4c-1767-4a0f-9a2c-80806e261889 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.860166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea328776-06c8-4d21-b9f3-5da80504d1ee admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.874071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570c6c3-dc33-440f-92f4-b64563095b9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.874854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570c6c3-dc33-440f-92f4-b64563095b9e service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.874992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d34a2c-b0d8-4496-a3f9-389d8be6aa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.875140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570c6c3-dc33-440f-92f4-b64563095b9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.875421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570c6c3-dc33-440f-92f4-b64563095b9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.875988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d34a2c-b0d8-4496-a3f9-389d8be6aa0b admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.876391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d34a2c-b0d8-4496-a3f9-389d8be6aa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.876677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d34a2c-b0d8-4496-a3f9-389d8be6aa0b admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.877461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d34a2c-b0d8-4496-a3f9-389d8be6aa0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.881196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d34a2c-b0d8-4496-a3f9-389d8be6aa0b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.901159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:22.911268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1050/2075] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:22 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:22.915600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d34a2c-b0d8-4496-a3f9-389d8be6aa0b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.936240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d34a2c-b0d8-4496-a3f9-389d8be6aa0b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:22.953216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e75605-057e-402d-ad64-a942cb95b27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:22.954240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e75605-057e-402d-ad64-a942cb95b27b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:22.954722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e75605-057e-402d-ad64-a942cb95b27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:22.955575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e75605-057e-402d-ad64-a942cb95b27b admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:22.956922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e75605-057e-402d-ad64-a942cb95b27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.959339 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e75605-057e-402d-ad64-a942cb95b27b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.984831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1051/2076] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:22.984991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570c6c3-dc33-440f-92f4-b64563095b9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:22.986814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570c6c3-dc33-440f-92f4-b64563095b9e service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:22.989038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e75605-057e-402d-ad64-a942cb95b27b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:22.998688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1026/2077] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:22 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:23.003586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e75605-057e-402d-ad64-a942cb95b27b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:23.009040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18275ce7-73e7-4023-9720-0ba7e44412eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:23.010482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18275ce7-73e7-4023-9720-0ba7e44412eb admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.012498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18275ce7-73e7-4023-9720-0ba7e44412eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.013715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18275ce7-73e7-4023-9720-0ba7e44412eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.014602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18275ce7-73e7-4023-9720-0ba7e44412eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:23.018458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18275ce7-73e7-4023-9720-0ba7e44412eb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:23.037025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1052/2078] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:23.038308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570c6c3-dc33-440f-92f4-b64563095b9e service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:23.051602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570c6c3-dc33-440f-92f4-b64563095b9e service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:23.060245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5d43-7c4e-4534-ad6d-8889660ba09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:23.061732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5d43-7c4e-4534-ad6d-8889660ba09e admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.062062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5d43-7c4e-4534-ad6d-8889660ba09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.062361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5d43-7c4e-4534-ad6d-8889660ba09e admin admin] PATH_INFO: `/v3/projects/0821e05ec3104167824698d17c677ad8/users/e71e5d6fcf514a27995ed9a81d11e593/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.095113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c5d43-7c4e-4534-ad6d-8889660ba09e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0821e05ec3104167824698d17c677ad8, user_id=e71e5d6fcf514a27995ed9a81d11e593, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:23.099257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c5d43-7c4e-4534-ad6d-8889660ba09e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:23.123264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1027/2079] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:23 2022] PUT /identity/v3/projects/0821e05ec3104167824698d17c677ad8/users/e71e5d6fcf514a27995ed9a81d11e593/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:23.134602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396fe2-6035-4bdd-9cfa-13e427ac81cd None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.134921 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396fe2-6035-4bdd-9cfa-13e427ac81cd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.135182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396fe2-6035-4bdd-9cfa-13e427ac81cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.211087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4a396fe2-6035-4bdd-9cfa-13e427ac81cd None None] MFA Rules not processed for user `e71e5d6fcf514a27995ed9a81d11e593`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:23.212319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4a396fe2-6035-4bdd-9cfa-13e427ac81cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:23.251125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5d43-7c4e-4534-ad6d-8889660ba09e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:23.297302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1053/2080] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:23.302023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396fe2-6035-4bdd-9cfa-13e427ac81cd None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:23.337426 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5d43-7c4e-4534-ad6d-8889660ba09e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:23.341997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4629f-ffe4-4b97-abc4-4908f7273fb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:23.342848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4629f-ffe4-4b97-abc4-4908f7273fb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.343460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4629f-ffe4-4b97-abc4-4908f7273fb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.343749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4629f-ffe4-4b97-abc4-4908f7273fb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.345734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-10b4629f-ffe4-4b97-abc4-4908f7273fb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:23.395383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396fe2-6035-4bdd-9cfa-13e427ac81cd None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:23.402996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b7318-2f42-45c0-9196-592817326b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:23.403722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b7318-2f42-45c0-9196-592817326b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.404072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b7318-2f42-45c0-9196-592817326b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.404415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b7318-2f42-45c0-9196-592817326b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.405367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b7318-2f42-45c0-9196-592817326b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:23.409043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b7318-2f42-45c0-9196-592817326b9a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:23.455875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1054/2081] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:23 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:31:23.458989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b7318-2f42-45c0-9196-592817326b9a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:23.481959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4629f-ffe4-4b97-abc4-4908f7273fb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:23.483620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4629f-ffe4-4b97-abc4-4908f7273fb9 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:23.490917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:23.494896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1028/2082] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:31:23.562864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b7318-2f42-45c0-9196-592817326b9a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:23.568209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ea9c7-4078-4184-8d8a-19c1290f4f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:23.569012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ea9c7-4078-4184-8d8a-19c1290f4f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.569294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ea9c7-4078-4184-8d8a-19c1290f4f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.569570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ea9c7-4078-4184-8d8a-19c1290f4f3f admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.570283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ea9c7-4078-4184-8d8a-19c1290f4f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:23.572266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ea9c7-4078-4184-8d8a-19c1290f4f3f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:23.606382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1055/2083] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:23 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:23.613031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4629f-ffe4-4b97-abc4-4908f7273fb9 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:23.626700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4629f-ffe4-4b97-abc4-4908f7273fb9 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:23.631609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9312a-9534-4b61-9879-26d110a4e6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:23.632371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9312a-9534-4b61-9879-26d110a4e6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.632748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9312a-9534-4b61-9879-26d110a4e6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.633122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9312a-9534-4b61-9879-26d110a4e6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.633935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9312a-9534-4b61-9879-26d110a4e6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:23.635809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9312a-9534-4b61-9879-26d110a4e6e1 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:23.647895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1029/2084] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:23.652912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ea9c7-4078-4184-8d8a-19c1290f4f3f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:23.671456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ea9c7-4078-4184-8d8a-19c1290f4f3f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:23.681495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747d586-5e58-43fa-832c-f0abecbabbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:23.682119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747d586-5e58-43fa-832c-f0abecbabbaa admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.682404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747d586-5e58-43fa-832c-f0abecbabbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.682679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747d586-5e58-43fa-832c-f0abecbabbaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.683606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747d586-5e58-43fa-832c-f0abecbabbaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:23.686478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747d586-5e58-43fa-832c-f0abecbabbaa admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:23.726262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1056/2085] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:23.728025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9312a-9534-4b61-9879-26d110a4e6e1 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:23.740677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9312a-9534-4b61-9879-26d110a4e6e1 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:23.745755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627dd7b-5323-482a-aafb-8ba991091dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:23.746651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627dd7b-5323-482a-aafb-8ba991091dec admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.746941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627dd7b-5323-482a-aafb-8ba991091dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.747220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627dd7b-5323-482a-aafb-8ba991091dec admin admin] PATH_INFO: `/v3/projects/05cee37c66294b25af6eff2549244d2b/users/a5de8a22ea8e4323a5239e837758ee76/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.764272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627dd7b-5323-482a-aafb-8ba991091dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=05cee37c66294b25af6eff2549244d2b, user_id=a5de8a22ea8e4323a5239e837758ee76, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:23.767511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627dd7b-5323-482a-aafb-8ba991091dec admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:23.789897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1030/2086] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:23 2022] PUT /identity/v3/projects/05cee37c66294b25af6eff2549244d2b/users/a5de8a22ea8e4323a5239e837758ee76/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:23.802114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de43043-b061-4d1a-ab5a-db14fef9552b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:23.802581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de43043-b061-4d1a-ab5a-db14fef9552b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:23.803354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de43043-b061-4d1a-ab5a-db14fef9552b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:23.848074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4de43043-b061-4d1a-ab5a-db14fef9552b None None] MFA Rules not processed for user `a5de8a22ea8e4323a5239e837758ee76`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:23.849391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4de43043-b061-4d1a-ab5a-db14fef9552b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:23.937127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1057/2087] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:23.941419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627dd7b-5323-482a-aafb-8ba991091dec admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:23.961860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:24.067813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627dd7b-5323-482a-aafb-8ba991091dec admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:24.074024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061739-9d4c-4e4b-937a-f200cc384756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:24.075006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061739-9d4c-4e4b-937a-f200cc384756 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:24.075825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061739-9d4c-4e4b-937a-f200cc384756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:24.075825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061739-9d4c-4e4b-937a-f200cc384756 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:24.076704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9061739-9d4c-4e4b-937a-f200cc384756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:24.078836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9061739-9d4c-4e4b-937a-f200cc384756 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:24.114521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de43043-b061-4d1a-ab5a-db14fef9552b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:24.116182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1031/2088] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:23 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:24.128214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de43043-b061-4d1a-ab5a-db14fef9552b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:24.202790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615bbf9-2ff2-47bc-8afd-b279b460bfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:24.203503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615bbf9-2ff2-47bc-8afd-b279b460bfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:24.203788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615bbf9-2ff2-47bc-8afd-b279b460bfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:24.204064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615bbf9-2ff2-47bc-8afd-b279b460bfd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:24.204761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615bbf9-2ff2-47bc-8afd-b279b460bfd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:24.206831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615bbf9-2ff2-47bc-8afd-b279b460bfd1 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:24.242503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1058/2089] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:24 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:24.247082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061739-9d4c-4e4b-937a-f200cc384756 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:24.258302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061739-9d4c-4e4b-937a-f200cc384756 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:24.269737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa62107-c6d2-44d3-a9ea-8264661342a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:24.270804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa62107-c6d2-44d3-a9ea-8264661342a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:24.279502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa62107-c6d2-44d3-a9ea-8264661342a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:24.280016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa62107-c6d2-44d3-a9ea-8264661342a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:24.281013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa62107-c6d2-44d3-a9ea-8264661342a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:24.282839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa62107-c6d2-44d3-a9ea-8264661342a4 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:24.304390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1032/2090] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:24.309260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615bbf9-2ff2-47bc-8afd-b279b460bfd1 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:24.320489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615bbf9-2ff2-47bc-8afd-b279b460bfd1 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:24.326456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21ea28-2518-4d0a-9846-53e5c079ae6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:24.327463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21ea28-2518-4d0a-9846-53e5c079ae6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:24.327751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21ea28-2518-4d0a-9846-53e5c079ae6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:24.328029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21ea28-2518-4d0a-9846-53e5c079ae6d admin admin] PATH_INFO: `/v3/projects/83aba09d6bb24b33a16e3ba21e6cfaf8/users/85f021c7519143c6aebfd69bc142d9d6/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:24.343139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:24.347711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21ea28-2518-4d0a-9846-53e5c079ae6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=83aba09d6bb24b33a16e3ba21e6cfaf8, user_id=85f021c7519143c6aebfd69bc142d9d6, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:24.351157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21ea28-2518-4d0a-9846-53e5c079ae6d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:24.381357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1059/2091] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:24 2022] PUT /identity/v3/projects/83aba09d6bb24b33a16e3ba21e6cfaf8/users/85f021c7519143c6aebfd69bc142d9d6/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:24.382392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa62107-c6d2-44d3-a9ea-8264661342a4 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:24.394347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa62107-c6d2-44d3-a9ea-8264661342a4 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:24.468591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0c584-4117-4977-a29e-678b14719d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:24.469402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0c584-4117-4977-a29e-678b14719d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:24.469760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0c584-4117-4977-a29e-678b14719d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:24.470743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0c584-4117-4977-a29e-678b14719d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:24.471876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0c584-4117-4977-a29e-678b14719d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:24.473724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0c584-4117-4977-a29e-678b14719d71 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:24.488704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1033/2092] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:24.489985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21ea28-2518-4d0a-9846-53e5c079ae6d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:24.509309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21ea28-2518-4d0a-9846-53e5c079ae6d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:24.519527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45262e-f50d-441b-8166-c14e5f9d05a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:24.521143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45262e-f50d-441b-8166-c14e5f9d05a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:24.522101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45262e-f50d-441b-8166-c14e5f9d05a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:24.522581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45262e-f50d-441b-8166-c14e5f9d05a5 admin admin] PATH_INFO: `/v3/projects/83aba09d6bb24b33a16e3ba21e6cfaf8/users/85f021c7519143c6aebfd69bc142d9d6/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:24.527031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45262e-f50d-441b-8166-c14e5f9d05a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83aba09d6bb24b33a16e3ba21e6cfaf8, user_id=85f021c7519143c6aebfd69bc142d9d6, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:24.531233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45262e-f50d-441b-8166-c14e5f9d05a5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:24.551499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1060/2093] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:24 2022] PUT /identity/v3/projects/83aba09d6bb24b33a16e3ba21e6cfaf8/users/85f021c7519143c6aebfd69bc142d9d6/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:24.561986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f695cec-a2bc-4617-9682-b303795a448e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:24.562275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f695cec-a2bc-4617-9682-b303795a448e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:24.562558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f695cec-a2bc-4617-9682-b303795a448e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:24.606454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-6f695cec-a2bc-4617-9682-b303795a448e None None] MFA Rules not processed for user `85f021c7519143c6aebfd69bc142d9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:24.608027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6f695cec-a2bc-4617-9682-b303795a448e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:24.734410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1034/2094] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:24.747330 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45262e-f50d-441b-8166-c14e5f9d05a5 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:24.888813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45262e-f50d-441b-8166-c14e5f9d05a5 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:24.894373 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca2ba2-0ad8-4c47-80af-ed11bac44fe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:24.895234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca2ba2-0ad8-4c47-80af-ed11bac44fe2 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:24.895976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca2ba2-0ad8-4c47-80af-ed11bac44fe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:24.896529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca2ba2-0ad8-4c47-80af-ed11bac44fe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:24.986009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca2ba2-0ad8-4c47-80af-ed11bac44fe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:24.987472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca2ba2-0ad8-4c47-80af-ed11bac44fe2 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:25.000395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1061/2095] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:26.530958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756645b5-9414-49ff-994e-f853c0100684 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:26.531453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756645b5-9414-49ff-994e-f853c0100684 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:26.531553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756645b5-9414-49ff-994e-f853c0100684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:26.539266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:26.561833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-756645b5-9414-49ff-994e-f853c0100684 None None] MFA Rules not processed for user `ceed468da0c844b688f5f91768b0dd96`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:26.563052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-756645b5-9414-49ff-994e-f853c0100684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:26.637663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1035/2096] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:26.646400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca2ba2-0ad8-4c47-80af-ed11bac44fe2 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:26.647945 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-67ca2ba2-0ad8-4c47-80af-ed11bac44fe2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:26.716212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca2ba2-0ad8-4c47-80af-ed11bac44fe2 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:26.721202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdf6b4-1f8a-449b-aaf0-9491a98f9055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:26.721885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdf6b4-1f8a-449b-aaf0-9491a98f9055 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:26.722175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdf6b4-1f8a-449b-aaf0-9491a98f9055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:26.722455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdf6b4-1f8a-449b-aaf0-9491a98f9055 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:26.723180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdf6b4-1f8a-449b-aaf0-9491a98f9055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:26.725212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdf6b4-1f8a-449b-aaf0-9491a98f9055 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:26.756183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1062/2097] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:26 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:26.757350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756645b5-9414-49ff-994e-f853c0100684 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:26.769424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756645b5-9414-49ff-994e-f853c0100684 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:26.827204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04f13b-6b44-4f88-bae2-3c5887c2dbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:26.827932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04f13b-6b44-4f88-bae2-3c5887c2dbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:26.828219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04f13b-6b44-4f88-bae2-3c5887c2dbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:26.828490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04f13b-6b44-4f88-bae2-3c5887c2dbc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:26.829262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04f13b-6b44-4f88-bae2-3c5887c2dbc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:26.831230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04f13b-6b44-4f88-bae2-3c5887c2dbc2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:26.852156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1036/2098] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:26 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:26.856835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdf6b4-1f8a-449b-aaf0-9491a98f9055 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:26.867840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdf6b4-1f8a-449b-aaf0-9491a98f9055 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:26.873743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5d068-24a5-48cf-afab-63e0ffdf62a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:26.874492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5d068-24a5-48cf-afab-63e0ffdf62a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:26.874843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5d068-24a5-48cf-afab-63e0ffdf62a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:26.875207 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5d068-24a5-48cf-afab-63e0ffdf62a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:26.876058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5d068-24a5-48cf-afab-63e0ffdf62a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:26.877710 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5d068-24a5-48cf-afab-63e0ffdf62a2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:26.888973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1063/2099] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:26.893325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04f13b-6b44-4f88-bae2-3c5887c2dbc2 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:26.903658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04f13b-6b44-4f88-bae2-3c5887c2dbc2 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:26.908505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a350a2-ff1c-407e-9efa-1ca12d393164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:26.909423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a350a2-ff1c-407e-9efa-1ca12d393164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:26.909704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a350a2-ff1c-407e-9efa-1ca12d393164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:26.910115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a350a2-ff1c-407e-9efa-1ca12d393164 admin admin] PATH_INFO: `/v3/projects/06253b6d930e47fc97c4a4c0376e0d5e/users/df1d0eecdec24af7ab1bab9c2b825a85/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:26.926945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a350a2-ff1c-407e-9efa-1ca12d393164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06253b6d930e47fc97c4a4c0376e0d5e, user_id=df1d0eecdec24af7ab1bab9c2b825a85, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:26.929678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a350a2-ff1c-407e-9efa-1ca12d393164 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:26.944288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1037/2100] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:26 2022] PUT /identity/v3/projects/06253b6d930e47fc97c4a4c0376e0d5e/users/df1d0eecdec24af7ab1bab9c2b825a85/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:26.948225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5d068-24a5-48cf-afab-63e0ffdf62a2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:26.955025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:26.959184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5d068-24a5-48cf-afab-63e0ffdf62a2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:27.026354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1007a-4736-481f-b9c0-ce5984d46651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:27.027651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1007a-4736-481f-b9c0-ce5984d46651 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:27.027967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1007a-4736-481f-b9c0-ce5984d46651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:27.028214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1007a-4736-481f-b9c0-ce5984d46651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:27.028971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1007a-4736-481f-b9c0-ce5984d46651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:27.030557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1007a-4736-481f-b9c0-ce5984d46651 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:27.040478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1064/2101] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:27.044737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a350a2-ff1c-407e-9efa-1ca12d393164 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:27.055398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a350a2-ff1c-407e-9efa-1ca12d393164 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:27.060118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c8404-aa97-43ec-b840-3e6d026ce034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:27.061050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c8404-aa97-43ec-b840-3e6d026ce034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:27.061335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c8404-aa97-43ec-b840-3e6d026ce034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:27.061610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c8404-aa97-43ec-b840-3e6d026ce034 admin admin] PATH_INFO: `/v3/projects/06253b6d930e47fc97c4a4c0376e0d5e/users/df1d0eecdec24af7ab1bab9c2b825a85/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:27.063596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c8404-aa97-43ec-b840-3e6d026ce034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06253b6d930e47fc97c4a4c0376e0d5e, user_id=df1d0eecdec24af7ab1bab9c2b825a85, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:27.066337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c8404-aa97-43ec-b840-3e6d026ce034 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:27.077657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1038/2102] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:27 2022] PUT /identity/v3/projects/06253b6d930e47fc97c4a4c0376e0d5e/users/df1d0eecdec24af7ab1bab9c2b825a85/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:30.330203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1007a-4736-481f-b9c0-ce5984d46651 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:30.442059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1007a-4736-481f-b9c0-ce5984d46651 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:30.567331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc39bc-c308-4492-a847-8fec7de76aac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:30.569363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc39bc-c308-4492-a847-8fec7de76aac service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:30.569812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc39bc-c308-4492-a847-8fec7de76aac service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:30.570238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc39bc-c308-4492-a847-8fec7de76aac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:30.722067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc39bc-c308-4492-a847-8fec7de76aac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:30.724560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc39bc-c308-4492-a847-8fec7de76aac service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:30.748033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1065/2103] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:31.875044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c8404-aa97-43ec-b840-3e6d026ce034 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:31.891921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c8404-aa97-43ec-b840-3e6d026ce034 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:31.914331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502a334-c436-4b11-8f7b-983721a08737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:31.915034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502a334-c436-4b11-8f7b-983721a08737 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:31.915317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502a334-c436-4b11-8f7b-983721a08737 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:31.915592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502a334-c436-4b11-8f7b-983721a08737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:32.024560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:32.057835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502a334-c436-4b11-8f7b-983721a08737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:32.060673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502a334-c436-4b11-8f7b-983721a08737 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:32.073678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1039/2104] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:32.906862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e285254-a9b9-4374-b0fc-cce2744e3c54 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:32.907360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e285254-a9b9-4374-b0fc-cce2744e3c54 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:32.907462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e285254-a9b9-4374-b0fc-cce2744e3c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:32.932219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:32.945203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1e285254-a9b9-4374-b0fc-cce2744e3c54 None None] MFA Rules not processed for user `df1d0eecdec24af7ab1bab9c2b825a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:32.946547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1e285254-a9b9-4374-b0fc-cce2744e3c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:33.032387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1066/2105] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:33.046717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502a334-c436-4b11-8f7b-983721a08737 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:33.048369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a502a334-c436-4b11-8f7b-983721a08737 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:33.136267 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502a334-c436-4b11-8f7b-983721a08737 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:33.140971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7364f-164d-4e29-ac29-ae25c2382fa9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb0e53d4b3774959820ac72824b3d4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:33.141644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7364f-164d-4e29-ac29-ae25c2382fa9 service glance] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:33.141977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7364f-164d-4e29-ac29-ae25c2382fa9 service glance] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:33.142257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7364f-164d-4e29-ac29-ae25c2382fa9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:33.153681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e285254-a9b9-4374-b0fc-cce2744e3c54 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:33.219959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7364f-164d-4e29-ac29-ae25c2382fa9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:33.221502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7364f-164d-4e29-ac29-ae25c2382fa9 service glance] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:33.267297 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1040/2106] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:33.269614 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e285254-a9b9-4374-b0fc-cce2744e3c54 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:33.274572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:33.275400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:33.275686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:33.275962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] PATH_INFO: `/v3/users/477dc35a7cc547508b9bafebc7581ad8` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:33.277419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=477dc35a7cc547508b9bafebc7581ad8)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:33.279687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:33.397625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '477dc35a7cc547508b9bafebc7581ad8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:33.452095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '477dc35a7cc547508b9bafebc7581ad8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:33.465862 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d9bd0-1086-4419-9b9e-e4d9012bd40c None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:33.466223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d9bd0-1086-4419-9b9e-e4d9012bd40c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:33.466537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d9bd0-1086-4419-9b9e-e4d9012bd40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:33.470497 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '477dc35a7cc547508b9bafebc7581ad8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:33.471308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '477dc35a7cc547508b9bafebc7581ad8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:33.472152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3bf45b4a-d66a-4bc2-bd4c-65b6ac5b1ce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '477dc35a7cc547508b9bafebc7581ad8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:33.491008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1067/2107] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:33 2022] DELETE /identity/v3/users/477dc35a7cc547508b9bafebc7581ad8 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:33.492786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117ed1d-b123-4619-8179-a60ab90233b4 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:33.493240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117ed1d-b123-4619-8179-a60ab90233b4 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:33.493653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117ed1d-b123-4619-8179-a60ab90233b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:33.501187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8b9d9bd0-1086-4419-9b9e-e4d9012bd40c None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:33.503096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8b9d9bd0-1086-4419-9b9e-e4d9012bd40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:33.516468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4117ed1d-b123-4619-8179-a60ab90233b4 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:33.518301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4117ed1d-b123-4619-8179-a60ab90233b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:33.537522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:33.606942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1041/2108] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:33.630677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1068/2109] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:33.641875 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d9bd0-1086-4419-9b9e-e4d9012bd40c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:33.644075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8b9d9bd0-1086-4419-9b9e-e4d9012bd40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:33.670469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117ed1d-b123-4619-8179-a60ab90233b4 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:33.672072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4117ed1d-b123-4619-8179-a60ab90233b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:33.743643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:33.779193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d9bd0-1086-4419-9b9e-e4d9012bd40c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:33.779193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d3796-d538-4e4f-979a-4b554bb3b360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:33.779193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d3796-d538-4e4f-979a-4b554bb3b360 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:33.779193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d3796-d538-4e4f-979a-4b554bb3b360 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:33.779676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d3796-d538-4e4f-979a-4b554bb3b360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:33.787921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d3796-d538-4e4f-979a-4b554bb3b360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:33.789618 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d3796-d538-4e4f-979a-4b554bb3b360 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:33.812478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1042/2110] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:33.841239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117ed1d-b123-4619-8179-a60ab90233b4 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:33.848585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925ffbc-349e-4e97-aa2c-1eb3ee9f6b29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:33.849516 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925ffbc-349e-4e97-aa2c-1eb3ee9f6b29 service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:33.849810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925ffbc-349e-4e97-aa2c-1eb3ee9f6b29 service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:33.850069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925ffbc-349e-4e97-aa2c-1eb3ee9f6b29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:33.875538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925ffbc-349e-4e97-aa2c-1eb3ee9f6b29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:33.877345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925ffbc-349e-4e97-aa2c-1eb3ee9f6b29 service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:33.917646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1069/2111] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:31:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 15:31:34.258301 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d3796-d538-4e4f-979a-4b554bb3b360 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:34.260537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-707d3796-d538-4e4f-979a-4b554bb3b360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:34.302400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d3796-d538-4e4f-979a-4b554bb3b360 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:34.310374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:34.311380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:34.311534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:34.311818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] PATH_INFO: `/v3/projects/f161514ca06b41e3898674a8be756209` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:34.313167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f161514ca06b41e3898674a8be756209)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:34.319185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:34.383481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f161514ca06b41e3898674a8be756209', 'request_id': 'req-5605588e-34ab-4513-acf1-b705e1d7cb3b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.392658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f161514ca06b41e3898674a8be756209', 'request_id': 'req-5605588e-34ab-4513-acf1-b705e1d7cb3b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.405810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f161514ca06b41e3898674a8be756209', 'request_id': 'req-5605588e-34ab-4513-acf1-b705e1d7cb3b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.420362 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] The token cache is being invalidate because project f161514ca06b41e3898674a8be756209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:34.420760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.421553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.423090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1043/2112] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:34 2022] DELETE /identity/v3/projects/f161514ca06b41e3898674a8be756209 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:34.428737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925ffbc-349e-4e97-aa2c-1eb3ee9f6b29 service swift] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:34.442447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925ffbc-349e-4e97-aa2c-1eb3ee9f6b29 service swift] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:34.525015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:34.525015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:34.525015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:34.525015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] PATH_INFO: `/v3/users/61a734cc31a94665a223fc1524dbb165` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:34.529329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a734cc31a94665a223fc1524dbb165)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:34.537938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:34.659019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a734cc31a94665a223fc1524dbb165'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.660312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:34.680167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a734cc31a94665a223fc1524dbb165'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.694727 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a734cc31a94665a223fc1524dbb165'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.695913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a734cc31a94665a223fc1524dbb165'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.697035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec40ea4f-6cd6-443a-93a0-884e4f0b83a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a734cc31a94665a223fc1524dbb165'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:34.714165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1070/2113] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:34 2022] DELETE /identity/v3/users/61a734cc31a94665a223fc1524dbb165 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:35.139101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:35.141508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:35.186144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605588e-34ab-4513-acf1-b705e1d7cb3b admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:35.309407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:35.310992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:35.311493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:35.311983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] PATH_INFO: `/v3/projects/1c3b41651cff482fa45e954d594ef078` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:35.316668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3b41651cff482fa45e954d594ef078)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:35.319130 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:35.444552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3b41651cff482fa45e954d594ef078', 'request_id': 'req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:35.463105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3b41651cff482fa45e954d594ef078', 'request_id': 'req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:35.477826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3b41651cff482fa45e954d594ef078', 'request_id': 'req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:35.487690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:35.520361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] The token cache is being invalidate because project 1c3b41651cff482fa45e954d594ef078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:35.522376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:35.528577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b5c6c93d-df53-47a5-8bbe-6131bc12fd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:35.530743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1044/2114] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:35 2022] DELETE /identity/v3/projects/1c3b41651cff482fa45e954d594ef078 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:35.535740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f801b77-cb6b-4f48-837e-38ac1bcd2870 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:35.536763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f801b77-cb6b-4f48-837e-38ac1bcd2870 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:35.537235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f801b77-cb6b-4f48-837e-38ac1bcd2870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:35.569983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-3f801b77-cb6b-4f48-837e-38ac1bcd2870 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:35.571547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3f801b77-cb6b-4f48-837e-38ac1bcd2870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:35.649748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1071/2115] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:35.657440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c9477-27bb-43b5-bf3f-ee93150c806b None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:35.658462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c9477-27bb-43b5-bf3f-ee93150c806b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:35.658847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c9477-27bb-43b5-bf3f-ee93150c806b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:35.686302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e87c9477-27bb-43b5-bf3f-ee93150c806b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:35.688286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e87c9477-27bb-43b5-bf3f-ee93150c806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:35.700944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1045/2116] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:35.708728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f801b77-cb6b-4f48-837e-38ac1bcd2870 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:35.727694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f801b77-cb6b-4f48-837e-38ac1bcd2870 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:35.734676 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f6fae-0c19-4ba5-9e38-48e95690972b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:35.735702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6fae-0c19-4ba5-9e38-48e95690972b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:35.735702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6fae-0c19-4ba5-9e38-48e95690972b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:35.735984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6fae-0c19-4ba5-9e38-48e95690972b admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:35.736772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f6fae-0c19-4ba5-9e38-48e95690972b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:35.737105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f6fae-0c19-4ba5-9e38-48e95690972b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:35.738974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f6fae-0c19-4ba5-9e38-48e95690972b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:35.757358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1072/2117] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:35.762903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c9477-27bb-43b5-bf3f-ee93150c806b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:35.772754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c9477-27bb-43b5-bf3f-ee93150c806b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:35.781959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9eb8be-beaf-478d-9791-b693d93b6fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:35.783120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9eb8be-beaf-478d-9791-b693d93b6fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:35.783359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9eb8be-beaf-478d-9791-b693d93b6fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:35.783807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9eb8be-beaf-478d-9791-b693d93b6fa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:35.785102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9eb8be-beaf-478d-9791-b693d93b6fa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:35.787083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9eb8be-beaf-478d-9791-b693d93b6fa5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:35.828126 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1046/2118] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:35 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:35.830461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f6fae-0c19-4ba5-9e38-48e95690972b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:35.845412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f6fae-0c19-4ba5-9e38-48e95690972b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:35.943972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cdc09-3c2e-47c5-ac9f-999fd3ecd462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:35.944734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cdc09-3c2e-47c5-ac9f-999fd3ecd462 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:35.945407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cdc09-3c2e-47c5-ac9f-999fd3ecd462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:35.945684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cdc09-3c2e-47c5-ac9f-999fd3ecd462 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:35.946451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cdc09-3c2e-47c5-ac9f-999fd3ecd462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:35.950700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cdc09-3c2e-47c5-ac9f-999fd3ecd462 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:35.982502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9eb8be-beaf-478d-9791-b693d93b6fa5 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:35.982828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1073/2119] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:35 2022] POST /identity/v3/users => generated 284 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:35.996209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9eb8be-beaf-478d-9791-b693d93b6fa5 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:36.009500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbaa12-e2fb-4d11-b35f-de16d09613f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:36.010186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbaa12-e2fb-4d11-b35f-de16d09613f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.010465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbaa12-e2fb-4d11-b35f-de16d09613f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.010739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbaa12-e2fb-4d11-b35f-de16d09613f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.012784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbaa12-e2fb-4d11-b35f-de16d09613f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:36.014444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbaa12-e2fb-4d11-b35f-de16d09613f5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:36.045117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1047/2120] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:36.049581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cdc09-3c2e-47c5-ac9f-999fd3ecd462 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:36.063931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cdc09-3c2e-47c5-ac9f-999fd3ecd462 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:36.072167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8951d7-d57b-4f48-a16b-5d5b8309803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:36.072883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8951d7-d57b-4f48-a16b-5d5b8309803c admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.073171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8951d7-d57b-4f48-a16b-5d5b8309803c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.073446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8951d7-d57b-4f48-a16b-5d5b8309803c admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.074728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8951d7-d57b-4f48-a16b-5d5b8309803c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:36.076342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8951d7-d57b-4f48-a16b-5d5b8309803c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:36.090482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1074/2121] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:36.094553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbaa12-e2fb-4d11-b35f-de16d09613f5 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:36.109187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbaa12-e2fb-4d11-b35f-de16d09613f5 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:36.118290 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc547da-6a19-4727-a464-8039df558f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:36.119289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc547da-6a19-4727-a464-8039df558f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.119970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc547da-6a19-4727-a464-8039df558f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.119970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc547da-6a19-4727-a464-8039df558f48 admin admin] PATH_INFO: `/v3/projects/014a76710d594940b1945af7a0c49fee/users/b04e03e796b141298a68868e0284388f/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.161335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc547da-6a19-4727-a464-8039df558f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014a76710d594940b1945af7a0c49fee, user_id=b04e03e796b141298a68868e0284388f, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:36.165226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc547da-6a19-4727-a464-8039df558f48 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:36.191609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1048/2122] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:36 2022] PUT /identity/v3/projects/014a76710d594940b1945af7a0c49fee/users/b04e03e796b141298a68868e0284388f/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:36.207778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226034c0-97ed-4c4c-9ecd-41415305d2d8 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.208034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226034c0-97ed-4c4c-9ecd-41415305d2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.208404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226034c0-97ed-4c4c-9ecd-41415305d2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.249452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-226034c0-97ed-4c4c-9ecd-41415305d2d8 None None] MFA Rules not processed for user `b04e03e796b141298a68868e0284388f`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:36.251399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-226034c0-97ed-4c4c-9ecd-41415305d2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:36.275381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:36.342130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1075/2123] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:36.346475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc547da-6a19-4727-a464-8039df558f48 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:36.468078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc547da-6a19-4727-a464-8039df558f48 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:36.475907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0696412-7666-4dc5-9ed0-ce0f4ee68fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:36.477036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0696412-7666-4dc5-9ed0-ce0f4ee68fdc admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.477494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0696412-7666-4dc5-9ed0-ce0f4ee68fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.478028 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0696412-7666-4dc5-9ed0-ce0f4ee68fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.479566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0696412-7666-4dc5-9ed0-ce0f4ee68fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:36.482637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0696412-7666-4dc5-9ed0-ce0f4ee68fdc admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:36.518741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1049/2124] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:36 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:36.522707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226034c0-97ed-4c4c-9ecd-41415305d2d8 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:36.537826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226034c0-97ed-4c4c-9ecd-41415305d2d8 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:36.638063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a207b-e9ad-45a7-9a41-b43db352291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:36.638754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a207b-e9ad-45a7-9a41-b43db352291f admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.639037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a207b-e9ad-45a7-9a41-b43db352291f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.639314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a207b-e9ad-45a7-9a41-b43db352291f admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.640017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a207b-e9ad-45a7-9a41-b43db352291f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:36.644257 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a207b-e9ad-45a7-9a41-b43db352291f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:36.651019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0696412-7666-4dc5-9ed0-ce0f4ee68fdc admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:36.655114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b0696412-7666-4dc5-9ed0-ce0f4ee68fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:36.672614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0696412-7666-4dc5-9ed0-ce0f4ee68fdc admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:36.678399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1076/2125] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:36 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:31:36.682851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a207b-e9ad-45a7-9a41-b43db352291f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:36.691416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:36.692401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.692692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.693023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] PATH_INFO: `/v3/users/e71e5d6fcf514a27995ed9a81d11e593` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.695075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71e5d6fcf514a27995ed9a81d11e593)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:36.697222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:36.713116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a207b-e9ad-45a7-9a41-b43db352291f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:36.725899 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61f79a-0028-4537-9c69-c83f56c1707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:36.727892 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61f79a-0028-4537-9c69-c83f56c1707a admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.728229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61f79a-0028-4537-9c69-c83f56c1707a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.728581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61f79a-0028-4537-9c69-c83f56c1707a admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.730482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61f79a-0028-4537-9c69-c83f56c1707a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:36.732955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61f79a-0028-4537-9c69-c83f56c1707a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:36.763916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1077/2126] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:36.765399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61f79a-0028-4537-9c69-c83f56c1707a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:36.794993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61f79a-0028-4537-9c69-c83f56c1707a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:36.801471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71e5d6fcf514a27995ed9a81d11e593'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:36.810907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71e5d6fcf514a27995ed9a81d11e593'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:36.825669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71e5d6fcf514a27995ed9a81d11e593'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:36.826693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71e5d6fcf514a27995ed9a81d11e593'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:36.833891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea958240-29fb-4f06-9a2a-5527b57e5dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:36.834993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea958240-29fb-4f06-9a2a-5527b57e5dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.835278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea958240-29fb-4f06-9a2a-5527b57e5dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.835555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea958240-29fb-4f06-9a2a-5527b57e5dc5 admin admin] PATH_INFO: `/v3/projects/726ff65056054a069dc52a0bc2c5dde3/users/5659ffcfb2c646fdb61f20665bb95f19/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.855809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:36.866506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-48c16137-5136-492c-a825-ab024073ef02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71e5d6fcf514a27995ed9a81d11e593'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:36.914407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea958240-29fb-4f06-9a2a-5527b57e5dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726ff65056054a069dc52a0bc2c5dde3, user_id=5659ffcfb2c646fdb61f20665bb95f19, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:36.918254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1050/2127] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:36 2022] DELETE /identity/v3/users/e71e5d6fcf514a27995ed9a81d11e593 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:36.922712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea958240-29fb-4f06-9a2a-5527b57e5dc5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:36.923153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd3afc-bb19-41d7-b2be-344b1a4c5bff None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:36.923444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd3afc-bb19-41d7-b2be-344b1a4c5bff None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:36.923751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd3afc-bb19-41d7-b2be-344b1a4c5bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:36.957750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1078/2128] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:36 2022] PUT /identity/v3/projects/726ff65056054a069dc52a0bc2c5dde3/users/5659ffcfb2c646fdb61f20665bb95f19/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:36.961959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea958240-29fb-4f06-9a2a-5527b57e5dc5 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:36.963213 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-21cd3afc-bb19-41d7-b2be-344b1a4c5bff None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:36.965621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-21cd3afc-bb19-41d7-b2be-344b1a4c5bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:36.968227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ea958240-29fb-4f06-9a2a-5527b57e5dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:37.084332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1051/2129] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:37.095276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd3afc-bb19-41d7-b2be-344b1a4c5bff None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:37.096720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-21cd3afc-bb19-41d7-b2be-344b1a4c5bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:37.126862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea958240-29fb-4f06-9a2a-5527b57e5dc5 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:37.137274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8a9b4-5759-41a0-8f04-085912d8af84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:37.139621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8a9b4-5759-41a0-8f04-085912d8af84 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:37.141166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8a9b4-5759-41a0-8f04-085912d8af84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:37.141526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8a9b4-5759-41a0-8f04-085912d8af84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:37.142379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8a9b4-5759-41a0-8f04-085912d8af84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:37.145855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8a9b4-5759-41a0-8f04-085912d8af84 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:37.158463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1079/2130] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:37.163457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8a9b4-5759-41a0-8f04-085912d8af84 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:37.176316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8a9b4-5759-41a0-8f04-085912d8af84 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:37.187234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b84c12-1138-40ec-a277-20a9b62755d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:37.188405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b84c12-1138-40ec-a277-20a9b62755d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:37.189076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b84c12-1138-40ec-a277-20a9b62755d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:37.189543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b84c12-1138-40ec-a277-20a9b62755d2 admin admin] PATH_INFO: `/v3/projects/726ff65056054a069dc52a0bc2c5dde3/users/5659ffcfb2c646fdb61f20665bb95f19/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:37.191639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b84c12-1138-40ec-a277-20a9b62755d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726ff65056054a069dc52a0bc2c5dde3, user_id=5659ffcfb2c646fdb61f20665bb95f19, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:37.193925 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd3afc-bb19-41d7-b2be-344b1a4c5bff None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:37.195858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b84c12-1138-40ec-a277-20a9b62755d2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:37.198324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9efc4-d7c9-43a5-9b60-c25529572a53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:37.200248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9efc4-d7c9-43a5-9b60-c25529572a53 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:37.200789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9efc4-d7c9-43a5-9b60-c25529572a53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:37.201314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9efc4-d7c9-43a5-9b60-c25529572a53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:37.203328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:37.210455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1080/2131] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:37 2022] PUT /identity/v3/projects/726ff65056054a069dc52a0bc2c5dde3/users/5659ffcfb2c646fdb61f20665bb95f19/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:37.216507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9efc4-d7c9-43a5-9b60-c25529572a53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:37.218111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9efc4-d7c9-43a5-9b60-c25529572a53 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:37.223029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6c937-647b-4f65-9903-6f59b8252085 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:37.223275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6c937-647b-4f65-9903-6f59b8252085 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:37.223615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6c937-647b-4f65-9903-6f59b8252085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:37.249560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1052/2132] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:37.281032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-9ad6c937-647b-4f65-9903-6f59b8252085 None None] MFA Rules not processed for user `5659ffcfb2c646fdb61f20665bb95f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:37.282912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9ad6c937-647b-4f65-9903-6f59b8252085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:37.459267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1081/2133] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:37.480372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9efc4-d7c9-43a5-9b60-c25529572a53 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:37.482416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-00c9efc4-d7c9-43a5-9b60-c25529572a53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:37.572288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9efc4-d7c9-43a5-9b60-c25529572a53 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:37.696756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:37.723575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa0f9e-b4d9-4a84-b056-412f433e320f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:37.724523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa0f9e-b4d9-4a84-b056-412f433e320f service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:37.725273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa0f9e-b4d9-4a84-b056-412f433e320f service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:37.725273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa0f9e-b4d9-4a84-b056-412f433e320f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:37.758552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa0f9e-b4d9-4a84-b056-412f433e320f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:37.773093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa0f9e-b4d9-4a84-b056-412f433e320f service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:37.835673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1053/2134] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:37.887169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6c937-647b-4f65-9903-6f59b8252085 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:37.889544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9ad6c937-647b-4f65-9903-6f59b8252085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:37.986546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa0f9e-b4d9-4a84-b056-412f433e320f service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:37.998903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa0f9e-b4d9-4a84-b056-412f433e320f service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:38.007072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b65b0-e596-448a-aed9-61ff67fed70a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:38.010176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b65b0-e596-448a-aed9-61ff67fed70a service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:38.011288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b65b0-e596-448a-aed9-61ff67fed70a service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:38.013152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b65b0-e596-448a-aed9-61ff67fed70a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:38.022185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a79b65b0-e596-448a-aed9-61ff67fed70a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:38.056582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6c937-647b-4f65-9903-6f59b8252085 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:38.070446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:38.072973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:38.073826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:38.074713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] PATH_INFO: `/v3/projects/0821e05ec3104167824698d17c677ad8` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:38.079822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0821e05ec3104167824698d17c677ad8)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:38.084586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:38.202416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b65b0-e596-448a-aed9-61ff67fed70a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:38.206876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b65b0-e596-448a-aed9-61ff67fed70a service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:38.215060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0821e05ec3104167824698d17c677ad8', 'request_id': 'req-838e7551-5279-425e-a66c-2f4c7bf1c5d3'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.233225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0821e05ec3104167824698d17c677ad8', 'request_id': 'req-838e7551-5279-425e-a66c-2f4c7bf1c5d3'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.234536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1054/2135] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:31:38.260471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0821e05ec3104167824698d17c677ad8', 'request_id': 'req-838e7551-5279-425e-a66c-2f4c7bf1c5d3'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.283835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] The token cache is being invalidate because project 0821e05ec3104167824698d17c677ad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:38.284230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.284985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.286785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1082/2136] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:37 2022] DELETE /identity/v3/projects/0821e05ec3104167824698d17c677ad8 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:38.290489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b65b0-e596-448a-aed9-61ff67fed70a service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:38.306493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b65b0-e596-448a-aed9-61ff67fed70a service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:38.433921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:38.434823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:38.435105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:38.435384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] PATH_INFO: `/v3/users/a5de8a22ea8e4323a5239e837758ee76` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:38.437283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5de8a22ea8e4323a5239e837758ee76)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:38.444200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:38.614548 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5de8a22ea8e4323a5239e837758ee76'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.616051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5de8a22ea8e4323a5239e837758ee76'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.626345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5de8a22ea8e4323a5239e837758ee76'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.627216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5de8a22ea8e4323a5239e837758ee76'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.669384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5de8a22ea8e4323a5239e837758ee76'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:38.727755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1055/2137] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:38 2022] DELETE /identity/v3/users/a5de8a22ea8e4323a5239e837758ee76 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:39.328256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:39.330381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:39.367123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e7551-5279-425e-a66c-2f4c7bf1c5d3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:39.379077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:39.463875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:39.465159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:39.465554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:39.465837 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] PATH_INFO: `/v3/projects/05cee37c66294b25af6eff2549244d2b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:39.469270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05cee37c66294b25af6eff2549244d2b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:39.471402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:39.555096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05cee37c66294b25af6eff2549244d2b', 'request_id': 'req-eada46b2-3f20-497b-b78d-e78e7151699c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.564253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05cee37c66294b25af6eff2549244d2b', 'request_id': 'req-eada46b2-3f20-497b-b78d-e78e7151699c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.574864 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05cee37c66294b25af6eff2549244d2b', 'request_id': 'req-eada46b2-3f20-497b-b78d-e78e7151699c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.597025 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] The token cache is being invalidate because project 05cee37c66294b25af6eff2549244d2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:39.597329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.598014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.601085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1083/2138] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:39 2022] DELETE /identity/v3/projects/05cee37c66294b25af6eff2549244d2b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:39.603940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:39.605553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:39.618721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:39.650455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2715ca-f0fb-4a96-bc8c-85d0c5320672 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:39.780306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:39.781163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:39.781444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:39.781724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] PATH_INFO: `/v3/users/85f021c7519143c6aebfd69bc142d9d6` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:39.782961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f021c7519143c6aebfd69bc142d9d6)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:39.784940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:39.848569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f021c7519143c6aebfd69bc142d9d6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.849316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f021c7519143c6aebfd69bc142d9d6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.861258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f021c7519143c6aebfd69bc142d9d6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.862086 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f021c7519143c6aebfd69bc142d9d6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.880472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87f4b801-fba1-4448-95b4-32a7ddd9323f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f021c7519143c6aebfd69bc142d9d6'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:39.895921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1056/2139] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:39 2022] DELETE /identity/v3/users/85f021c7519143c6aebfd69bc142d9d6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:40.288960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:40.290907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:40.509683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada46b2-3f20-497b-b78d-e78e7151699c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:40.589239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:40.602417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:40.609984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:40.610374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:40.610771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] PATH_INFO: `/v3/projects/83aba09d6bb24b33a16e3ba21e6cfaf8` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:40.612287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=83aba09d6bb24b33a16e3ba21e6cfaf8)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:40.614561 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:40.700331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83aba09d6bb24b33a16e3ba21e6cfaf8', 'request_id': 'req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:40.712798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83aba09d6bb24b33a16e3ba21e6cfaf8', 'request_id': 'req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:40.721283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83aba09d6bb24b33a16e3ba21e6cfaf8', 'request_id': 'req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:40.739019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] The token cache is being invalidate because project 83aba09d6bb24b33a16e3ba21e6cfaf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:40.739625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:40.741648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-cda15e63-6f3b-4f53-ba36-5146a40a8bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:40.746335 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1084/2140] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:40 2022] DELETE /identity/v3/projects/83aba09d6bb24b33a16e3ba21e6cfaf8 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:40.751800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d1f60-3774-46f3-992f-f7eacbeb7b87 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:40.752283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d1f60-3774-46f3-992f-f7eacbeb7b87 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:40.752396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d1f60-3774-46f3-992f-f7eacbeb7b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:40.811612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:40.827152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-235d1f60-3774-46f3-992f-f7eacbeb7b87 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:40.828808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-235d1f60-3774-46f3-992f-f7eacbeb7b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:40.921713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1057/2141] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:40.923906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aa343-b9e6-4af5-8510-e6f15a4c3af3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:40.924194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aa343-b9e6-4af5-8510-e6f15a4c3af3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:40.924470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aa343-b9e6-4af5-8510-e6f15a4c3af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:40.952567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0b3aa343-b9e6-4af5-8510-e6f15a4c3af3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:40.954128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0b3aa343-b9e6-4af5-8510-e6f15a4c3af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:40.964900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1085/2142] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:40.969399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d1f60-3774-46f3-992f-f7eacbeb7b87 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:40.985727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d1f60-3774-46f3-992f-f7eacbeb7b87 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:40.990867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9067e-384b-4429-91c4-dbe4f4406d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:40.991703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9067e-384b-4429-91c4-dbe4f4406d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:40.992173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9067e-384b-4429-91c4-dbe4f4406d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:40.992599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9067e-384b-4429-91c4-dbe4f4406d35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:40.993635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9067e-384b-4429-91c4-dbe4f4406d35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:40.994096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9067e-384b-4429-91c4-dbe4f4406d35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:40.996002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9067e-384b-4429-91c4-dbe4f4406d35 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:41.009949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1058/2143] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:41.014671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aa343-b9e6-4af5-8510-e6f15a4c3af3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:41.024583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aa343-b9e6-4af5-8510-e6f15a4c3af3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:41.031934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947888f0-c49b-48b9-86a6-c9c57567af4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:41.032613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947888f0-c49b-48b9-86a6-c9c57567af4e admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:41.032928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947888f0-c49b-48b9-86a6-c9c57567af4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:41.033214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947888f0-c49b-48b9-86a6-c9c57567af4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:41.033931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947888f0-c49b-48b9-86a6-c9c57567af4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:41.035943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947888f0-c49b-48b9-86a6-c9c57567af4e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:41.071928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1086/2144] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:41 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:31:41.073195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9067e-384b-4429-91c4-dbe4f4406d35 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:41.091767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9067e-384b-4429-91c4-dbe4f4406d35 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:41.187277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df13fe3-9c00-4e62-bfb0-a5533b98e945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:41.188044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df13fe3-9c00-4e62-bfb0-a5533b98e945 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:41.188328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df13fe3-9c00-4e62-bfb0-a5533b98e945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:41.188604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df13fe3-9c00-4e62-bfb0-a5533b98e945 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:41.189348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df13fe3-9c00-4e62-bfb0-a5533b98e945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:41.192626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df13fe3-9c00-4e62-bfb0-a5533b98e945 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:41.237149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1059/2145] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:41 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:31:41.241926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947888f0-c49b-48b9-86a6-c9c57567af4e admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:41.261988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947888f0-c49b-48b9-86a6-c9c57567af4e admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:41.277840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4f5ca-ea3b-460c-a75e-f6e90618fa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:41.280114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4f5ca-ea3b-460c-a75e-f6e90618fa91 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:41.280986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4f5ca-ea3b-460c-a75e-f6e90618fa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:41.281287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4f5ca-ea3b-460c-a75e-f6e90618fa91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:41.282051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4f5ca-ea3b-460c-a75e-f6e90618fa91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:41.286713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4f5ca-ea3b-460c-a75e-f6e90618fa91 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:41.310353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1087/2146] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:41.313401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df13fe3-9c00-4e62-bfb0-a5533b98e945 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:41.335624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df13fe3-9c00-4e62-bfb0-a5533b98e945 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:41.345229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db118c68-4494-4cd2-be76-40fd27865ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:41.346064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118c68-4494-4cd2-be76-40fd27865ced admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:41.346633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118c68-4494-4cd2-be76-40fd27865ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:41.346923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118c68-4494-4cd2-be76-40fd27865ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:41.347937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db118c68-4494-4cd2-be76-40fd27865ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:41.353453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db118c68-4494-4cd2-be76-40fd27865ced admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:41.369332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1060/2147] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:41.374166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4f5ca-ea3b-460c-a75e-f6e90618fa91 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:41.397626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4f5ca-ea3b-460c-a75e-f6e90618fa91 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:41.405264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98370e7-e322-48a2-b792-d0bbd93ee8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:41.405872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98370e7-e322-48a2-b792-d0bbd93ee8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:41.406064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98370e7-e322-48a2-b792-d0bbd93ee8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:41.406343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98370e7-e322-48a2-b792-d0bbd93ee8a5 admin admin] PATH_INFO: `/v3/projects/d91b049126b7462ca8b5d0ed79a1c6ef/users/398779837495418fb9953b928209b88f/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:41.436136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98370e7-e322-48a2-b792-d0bbd93ee8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91b049126b7462ca8b5d0ed79a1c6ef, user_id=398779837495418fb9953b928209b88f, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:41.442863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98370e7-e322-48a2-b792-d0bbd93ee8a5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:41.479735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634de7b-25f5-4976-98a1-4bc5a0f6c554 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:41.480192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634de7b-25f5-4976-98a1-4bc5a0f6c554 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:41.480601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634de7b-25f5-4976-98a1-4bc5a0f6c554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:41.482285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1088/2148] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:41 2022] PUT /identity/v3/projects/d91b049126b7462ca8b5d0ed79a1c6ef/users/398779837495418fb9953b928209b88f/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:41.528981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-f634de7b-25f5-4976-98a1-4bc5a0f6c554 None None] MFA Rules not processed for user `398779837495418fb9953b928209b88f`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:41.531081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f634de7b-25f5-4976-98a1-4bc5a0f6c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:41.680646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:41.688728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1061/2149] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:41.702331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98370e7-e322-48a2-b792-d0bbd93ee8a5 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:41.704119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f98370e7-e322-48a2-b792-d0bbd93ee8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:41.821315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98370e7-e322-48a2-b792-d0bbd93ee8a5 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:41.832142 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf686b-614c-47c9-9875-006322ddf119 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb0e53d4b3774959820ac72824b3d4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:41.833171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf686b-614c-47c9-9875-006322ddf119 service glance] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:41.833588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf686b-614c-47c9-9875-006322ddf119 service glance] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:41.833874 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf686b-614c-47c9-9875-006322ddf119 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:41.847107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf686b-614c-47c9-9875-006322ddf119 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:41.848565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf686b-614c-47c9-9875-006322ddf119 service glance] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:41.859394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1089/2150] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:41 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:43.062861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634de7b-25f5-4976-98a1-4bc5a0f6c554 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:43.064438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f634de7b-25f5-4976-98a1-4bc5a0f6c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:43.163575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634de7b-25f5-4976-98a1-4bc5a0f6c554 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:43.169410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:43.170291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:43.170579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:43.170850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] PATH_INFO: `/v3/users/8d5be27c60a24e8083c99b1d1160c7b2` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:43.172193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d5be27c60a24e8083c99b1d1160c7b2)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:43.174434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:43.233843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d5be27c60a24e8083c99b1d1160c7b2'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:43.234694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d5be27c60a24e8083c99b1d1160c7b2'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:43.249959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d5be27c60a24e8083c99b1d1160c7b2'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:43.250855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d5be27c60a24e8083c99b1d1160c7b2'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:43.271978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d5be27c60a24e8083c99b1d1160c7b2'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:43.296135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1062/2151] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:43 2022] DELETE /identity/v3/users/8d5be27c60a24e8083c99b1d1160c7b2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:43.298247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5a8e2-b830-4807-b0ff-26f02ed024eb None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:43.299077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5a8e2-b830-4807-b0ff-26f02ed024eb None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:43.299512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5a8e2-b830-4807-b0ff-26f02ed024eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:43.314685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:43.329363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-7ed5a8e2-b830-4807-b0ff-26f02ed024eb None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:43.333897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7ed5a8e2-b830-4807-b0ff-26f02ed024eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:43.427446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1090/2152] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:43.438812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:43.440612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:43.598253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1d76a-df00-434d-938a-7f9a04f791b2 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:43.605728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746757d3-581d-491b-9420-e5140e159843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:43.607437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746757d3-581d-491b-9420-e5140e159843 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:43.608236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746757d3-581d-491b-9420-e5140e159843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:43.608764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746757d3-581d-491b-9420-e5140e159843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:43.622952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746757d3-581d-491b-9420-e5140e159843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:43.624522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746757d3-581d-491b-9420-e5140e159843 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:43.637907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1063/2153] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:44.025865 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5a8e2-b830-4807-b0ff-26f02ed024eb None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:44.028128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7ed5a8e2-b830-4807-b0ff-26f02ed024eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:44.067718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5a8e2-b830-4807-b0ff-26f02ed024eb None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:44.074762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:44.075645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.075926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.076295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] PATH_INFO: `/v3/projects/2f9d20f74acb4df68802f42512f81b02` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.077538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f9d20f74acb4df68802f42512f81b02)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:44.079521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:44.145186 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f9d20f74acb4df68802f42512f81b02', 'request_id': 'req-e4c2829f-d123-44be-af71-1c73a4f25948'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.166502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f9d20f74acb4df68802f42512f81b02', 'request_id': 'req-e4c2829f-d123-44be-af71-1c73a4f25948'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.177209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f9d20f74acb4df68802f42512f81b02', 'request_id': 'req-e4c2829f-d123-44be-af71-1c73a4f25948'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.198640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] The token cache is being invalidate because project 2f9d20f74acb4df68802f42512f81b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:44.199291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.200221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e4c2829f-d123-44be-af71-1c73a4f25948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.204931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1091/2154] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:44 2022] DELETE /identity/v3/projects/2f9d20f74acb4df68802f42512f81b02 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:44.208540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b94aa5-a6bf-4482-b567-93b927172272 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.208540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b94aa5-a6bf-4482-b567-93b927172272 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.208540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b94aa5-a6bf-4482-b567-93b927172272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.215809 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:44.234374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-94b94aa5-a6bf-4482-b567-93b927172272 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:44.235691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-94b94aa5-a6bf-4482-b567-93b927172272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:44.330657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1064/2155] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:44.332787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21d736-c586-4696-b6b5-e1771f6da82f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.333611 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21d736-c586-4696-b6b5-e1771f6da82f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.334063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21d736-c586-4696-b6b5-e1771f6da82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.371522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ea21d736-c586-4696-b6b5-e1771f6da82f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:44.373447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ea21d736-c586-4696-b6b5-e1771f6da82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:44.384531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b94aa5-a6bf-4482-b567-93b927172272 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:44.387894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1092/2156] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:44.391673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-94b94aa5-a6bf-4482-b567-93b927172272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:44.393483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21d736-c586-4696-b6b5-e1771f6da82f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:44.403908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21d736-c586-4696-b6b5-e1771f6da82f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:44.405985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b94aa5-a6bf-4482-b567-93b927172272 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:44.411142 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450c73a-32cb-45c0-9e96-94545fbf611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:44.411987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450c73a-32cb-45c0-9e96-94545fbf611b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.412423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450c73a-32cb-45c0-9e96-94545fbf611b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.415657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450c73a-32cb-45c0-9e96-94545fbf611b admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.416584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450c73a-32cb-45c0-9e96-94545fbf611b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:44.417080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450c73a-32cb-45c0-9e96-94545fbf611b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:44.419062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450c73a-32cb-45c0-9e96-94545fbf611b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:44.429670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:44.430681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.431064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.431449 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] PATH_INFO: `/v3/users/b04e03e796b141298a68868e0284388f` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.433918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04e03e796b141298a68868e0284388f)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:44.437006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1093/2157] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:44.437490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:44.441277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450c73a-32cb-45c0-9e96-94545fbf611b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:44.459139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450c73a-32cb-45c0-9e96-94545fbf611b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:44.470912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25153d9e-d349-46d4-8bce-a8d3b4215268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:44.471945 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25153d9e-d349-46d4-8bce-a8d3b4215268 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.472235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25153d9e-d349-46d4-8bce-a8d3b4215268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.472511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25153d9e-d349-46d4-8bce-a8d3b4215268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.473795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25153d9e-d349-46d4-8bce-a8d3b4215268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:44.475929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25153d9e-d349-46d4-8bce-a8d3b4215268 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:44.512746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1094/2158] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:44 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:31:44.513749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25153d9e-d349-46d4-8bce-a8d3b4215268 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:44.530644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04e03e796b141298a68868e0284388f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.531746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04e03e796b141298a68868e0284388f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.546805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04e03e796b141298a68868e0284388f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.549097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04e03e796b141298a68868e0284388f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.576358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-cde2c289-895b-45bd-a6aa-26b034911480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04e03e796b141298a68868e0284388f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:44.605082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1065/2159] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:44 2022] DELETE /identity/v3/users/b04e03e796b141298a68868e0284388f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:44.608674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ea5a1-fa6d-4f74-89de-203021c36808 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.609150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ea5a1-fa6d-4f74-89de-203021c36808 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.610322 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ea5a1-fa6d-4f74-89de-203021c36808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.625287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:44.647484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-012ea5a1-fa6d-4f74-89de-203021c36808 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:44.650281 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-012ea5a1-fa6d-4f74-89de-203021c36808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:44.658236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25153d9e-d349-46d4-8bce-a8d3b4215268 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:44.786934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca063c7-7633-40fc-bdfa-d52703bba096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:44.787745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca063c7-7633-40fc-bdfa-d52703bba096 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.788116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca063c7-7633-40fc-bdfa-d52703bba096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.788443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca063c7-7633-40fc-bdfa-d52703bba096 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.789342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca063c7-7633-40fc-bdfa-d52703bba096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:44.791917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca063c7-7633-40fc-bdfa-d52703bba096 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:44.800651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1066/2160] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:44.801885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ea5a1-fa6d-4f74-89de-203021c36808 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:44.803780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-012ea5a1-fa6d-4f74-89de-203021c36808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:44.824755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1095/2161] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:44 2022] POST /identity/v3/users => generated 285 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:44.826312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca063c7-7633-40fc-bdfa-d52703bba096 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:44.832833 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2ca063c7-7633-40fc-bdfa-d52703bba096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:44.835090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:44.849480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca063c7-7633-40fc-bdfa-d52703bba096 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:44.858858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c8cb3-8c85-4f8f-a3e9-bbc6db817d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:44.859631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c8cb3-8c85-4f8f-a3e9-bbc6db817d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.859997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c8cb3-8c85-4f8f-a3e9-bbc6db817d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.860412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c8cb3-8c85-4f8f-a3e9-bbc6db817d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.862053 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c8cb3-8c85-4f8f-a3e9-bbc6db817d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:44.863866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c8cb3-8c85-4f8f-a3e9-bbc6db817d22 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:44.877225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1096/2162] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:44.879381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c8cb3-8c85-4f8f-a3e9-bbc6db817d22 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:44.891023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c8cb3-8c85-4f8f-a3e9-bbc6db817d22 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:44.892011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ea5a1-fa6d-4f74-89de-203021c36808 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:44.896193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68518ec5-4409-4406-b9a0-eb0320c2c280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:44.896662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433ab20-7b65-4aed-bbee-76673e1951af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:44.897047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68518ec5-4409-4406-b9a0-eb0320c2c280 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.897354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433ab20-7b65-4aed-bbee-76673e1951af service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.897498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68518ec5-4409-4406-b9a0-eb0320c2c280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.897640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433ab20-7b65-4aed-bbee-76673e1951af service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.897787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68518ec5-4409-4406-b9a0-eb0320c2c280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.897918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433ab20-7b65-4aed-bbee-76673e1951af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.898624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68518ec5-4409-4406-b9a0-eb0320c2c280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:44.900288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68518ec5-4409-4406-b9a0-eb0320c2c280 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:44.911846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433ab20-7b65-4aed-bbee-76673e1951af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:44.913368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433ab20-7b65-4aed-bbee-76673e1951af service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:44.915347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1097/2163] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:44.919587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68518ec5-4409-4406-b9a0-eb0320c2c280 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:44.927800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1067/2164] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:44.932745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68518ec5-4409-4406-b9a0-eb0320c2c280 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:44.939079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830788bd-b40a-49d8-90df-3a2616fde783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:44.940653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830788bd-b40a-49d8-90df-3a2616fde783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.941066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830788bd-b40a-49d8-90df-3a2616fde783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.941424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830788bd-b40a-49d8-90df-3a2616fde783 admin admin] PATH_INFO: `/v3/projects/2d5867b29ac444959b10c0653ed1ef15/users/c5fe3a6de74c4402b1e2d7f9cab4852f/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:44.960630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830788bd-b40a-49d8-90df-3a2616fde783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5867b29ac444959b10c0653ed1ef15, user_id=c5fe3a6de74c4402b1e2d7f9cab4852f, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:44.963334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830788bd-b40a-49d8-90df-3a2616fde783 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:44.984964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1098/2165] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:44 2022] PUT /identity/v3/projects/2d5867b29ac444959b10c0653ed1ef15/users/c5fe3a6de74c4402b1e2d7f9cab4852f/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:44.990680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea9220-40b9-47ec-9178-b2aa48e8c455 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:44.991023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea9220-40b9-47ec-9178-b2aa48e8c455 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:44.991311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea9220-40b9-47ec-9178-b2aa48e8c455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:45.035153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8aea9220-40b9-47ec-9178-b2aa48e8c455 None None] MFA Rules not processed for user `c5fe3a6de74c4402b1e2d7f9cab4852f`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:45.037477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8aea9220-40b9-47ec-9178-b2aa48e8c455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:45.196405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1068/2166] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:45.286804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830788bd-b40a-49d8-90df-3a2616fde783 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:45.288801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-830788bd-b40a-49d8-90df-3a2616fde783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:45.371830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830788bd-b40a-49d8-90df-3a2616fde783 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:45.379682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:45.380657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:45.381068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:45.381456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] PATH_INFO: `/v3/projects/014a76710d594940b1945af7a0c49fee` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:45.382756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=014a76710d594940b1945af7a0c49fee)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:45.384802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:45.446821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:45.449840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014a76710d594940b1945af7a0c49fee', 'request_id': 'req-76794b46-48fd-42b9-bc53-a8a6dc014610'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.459358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014a76710d594940b1945af7a0c49fee', 'request_id': 'req-76794b46-48fd-42b9-bc53-a8a6dc014610'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.467305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014a76710d594940b1945af7a0c49fee', 'request_id': 'req-76794b46-48fd-42b9-bc53-a8a6dc014610'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.487250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] The token cache is being invalidate because project 014a76710d594940b1945af7a0c49fee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:45.487579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.488307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.490137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1099/2167] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:45 2022] DELETE /identity/v3/projects/014a76710d594940b1945af7a0c49fee => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:45.493961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea9220-40b9-47ec-9178-b2aa48e8c455 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:45.505795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea9220-40b9-47ec-9178-b2aa48e8c455 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:45.570835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:45.571843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:45.572216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:45.572585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] PATH_INFO: `/v3/users/5659ffcfb2c646fdb61f20665bb95f19` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:45.574616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5659ffcfb2c646fdb61f20665bb95f19)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:45.577036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:45.649433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5659ffcfb2c646fdb61f20665bb95f19'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.650076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5659ffcfb2c646fdb61f20665bb95f19'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.652287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:45.664740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5659ffcfb2c646fdb61f20665bb95f19'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.665543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5659ffcfb2c646fdb61f20665bb95f19'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.692159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-60bd2c35-b705-4d39-aeb7-54f1222f783e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5659ffcfb2c646fdb61f20665bb95f19'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:45.718670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1069/2168] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:45 2022] DELETE /identity/v3/users/5659ffcfb2c646fdb61f20665bb95f19 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:46.209629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:46.228298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:46.277324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76794b46-48fd-42b9-bc53-a8a6dc014610 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:46.437259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:46.438145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:46.438431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:46.438713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] PATH_INFO: `/v3/projects/726ff65056054a069dc52a0bc2c5dde3` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:46.440109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] RBAC: Authorizing `identity:delete_project(project_id=726ff65056054a069dc52a0bc2c5dde3)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:46.442124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:46.544944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726ff65056054a069dc52a0bc2c5dde3', 'request_id': 'req-0bb7e74d-c594-4098-996e-1b0a6276badc'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:46.553391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726ff65056054a069dc52a0bc2c5dde3', 'request_id': 'req-0bb7e74d-c594-4098-996e-1b0a6276badc'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:46.561345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726ff65056054a069dc52a0bc2c5dde3', 'request_id': 'req-0bb7e74d-c594-4098-996e-1b0a6276badc'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:46.591583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] The token cache is being invalidate because project 726ff65056054a069dc52a0bc2c5dde3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:46.592323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:46.593449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-0bb7e74d-c594-4098-996e-1b0a6276badc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:46.598482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1100/2169] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:46 2022] DELETE /identity/v3/projects/726ff65056054a069dc52a0bc2c5dde3 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:46.603241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad407e-50aa-4cf0-b693-57da98e60a62 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:46.603772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad407e-50aa-4cf0-b693-57da98e60a62 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:46.603772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad407e-50aa-4cf0-b693-57da98e60a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:46.637490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-35ad407e-50aa-4cf0-b693-57da98e60a62 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:46.641846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-35ad407e-50aa-4cf0-b693-57da98e60a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:46.744639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1070/2170] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:46.753633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f38aa5-4149-4977-a588-a0f6133ecc75 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:46.754027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f38aa5-4149-4977-a588-a0f6133ecc75 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:46.754416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f38aa5-4149-4977-a588-a0f6133ecc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:46.794208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-45f38aa5-4149-4977-a588-a0f6133ecc75 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:46.795883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-45f38aa5-4149-4977-a588-a0f6133ecc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:46.807105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1101/2171] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:46.811440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad407e-50aa-4cf0-b693-57da98e60a62 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:46.824021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad407e-50aa-4cf0-b693-57da98e60a62 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:46.830248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308e5c5-72ff-4ea6-be47-42a667bc33f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:46.830973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308e5c5-72ff-4ea6-be47-42a667bc33f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:46.831332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308e5c5-72ff-4ea6-be47-42a667bc33f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:46.831692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308e5c5-72ff-4ea6-be47-42a667bc33f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:46.832629 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308e5c5-72ff-4ea6-be47-42a667bc33f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:46.833111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308e5c5-72ff-4ea6-be47-42a667bc33f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:46.835198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308e5c5-72ff-4ea6-be47-42a667bc33f7 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:46.862023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1071/2172] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:31:46.863421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f38aa5-4149-4977-a588-a0f6133ecc75 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:46.873529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f38aa5-4149-4977-a588-a0f6133ecc75 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:46.878432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be7cf9-c2b1-47fb-90e2-6e0098be1c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:46.879134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be7cf9-c2b1-47fb-90e2-6e0098be1c89 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:46.879518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be7cf9-c2b1-47fb-90e2-6e0098be1c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:46.879695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be7cf9-c2b1-47fb-90e2-6e0098be1c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:46.880410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be7cf9-c2b1-47fb-90e2-6e0098be1c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:46.882343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be7cf9-c2b1-47fb-90e2-6e0098be1c89 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:46.914300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1102/2173] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:46 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:46.918799 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308e5c5-72ff-4ea6-be47-42a667bc33f7 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:46.929697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308e5c5-72ff-4ea6-be47-42a667bc33f7 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:47.014038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deee75-28aa-4006-a54d-467e6c51cc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:47.014716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deee75-28aa-4006-a54d-467e6c51cc23 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.014998 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deee75-28aa-4006-a54d-467e6c51cc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.015272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deee75-28aa-4006-a54d-467e6c51cc23 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.015977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7deee75-28aa-4006-a54d-467e6c51cc23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:47.018159 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7deee75-28aa-4006-a54d-467e6c51cc23 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:47.045801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1072/2174] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:46 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:31:47.050059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be7cf9-c2b1-47fb-90e2-6e0098be1c89 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:47.061628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be7cf9-c2b1-47fb-90e2-6e0098be1c89 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:47.070280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb37ed-14f3-4b1f-81f2-d4ce5c175afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:47.071063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb37ed-14f3-4b1f-81f2-d4ce5c175afc admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.071434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb37ed-14f3-4b1f-81f2-d4ce5c175afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.071764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb37ed-14f3-4b1f-81f2-d4ce5c175afc admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.072575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb37ed-14f3-4b1f-81f2-d4ce5c175afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:47.074850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb37ed-14f3-4b1f-81f2-d4ce5c175afc admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:47.096690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1103/2175] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:47.098230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deee75-28aa-4006-a54d-467e6c51cc23 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:47.109135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deee75-28aa-4006-a54d-467e6c51cc23 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:47.114080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1cde-a170-4e70-8461-da1aa739de25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:47.115390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1cde-a170-4e70-8461-da1aa739de25 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.115658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1cde-a170-4e70-8461-da1aa739de25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.115969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1cde-a170-4e70-8461-da1aa739de25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.116670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec1cde-a170-4e70-8461-da1aa739de25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:47.118339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec1cde-a170-4e70-8461-da1aa739de25 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:47.132965 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1073/2176] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:47.137662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb37ed-14f3-4b1f-81f2-d4ce5c175afc admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:47.157367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1cde-a170-4e70-8461-da1aa739de25 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:47.159071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-eeec1cde-a170-4e70-8461-da1aa739de25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:47.169985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb37ed-14f3-4b1f-81f2-d4ce5c175afc admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:47.178562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77364323-9b7f-4c52-9f04-719a49fe17ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:47.179464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77364323-9b7f-4c52-9f04-719a49fe17ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.179743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77364323-9b7f-4c52-9f04-719a49fe17ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.180018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77364323-9b7f-4c52-9f04-719a49fe17ac admin admin] PATH_INFO: `/v3/projects/97995b2cf66948f4984ff7804e90a431/users/b27d09f23f374e5c93b84fd8e263c788/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.204134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77364323-9b7f-4c52-9f04-719a49fe17ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=97995b2cf66948f4984ff7804e90a431, user_id=b27d09f23f374e5c93b84fd8e263c788, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:47.209163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77364323-9b7f-4c52-9f04-719a49fe17ac admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:47.212093 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:47.215600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:47.241315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1104/2177] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:47 2022] PUT /identity/v3/projects/97995b2cf66948f4984ff7804e90a431/users/b27d09f23f374e5c93b84fd8e263c788/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:47.246931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47602e3a-54bd-45e3-b3e5-eca48cecb916 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.247519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47602e3a-54bd-45e3-b3e5-eca48cecb916 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.247951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47602e3a-54bd-45e3-b3e5-eca48cecb916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.268085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1cde-a170-4e70-8461-da1aa739de25 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:47.280087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-47602e3a-54bd-45e3-b3e5-eca48cecb916 None None] MFA Rules not processed for user `b27d09f23f374e5c93b84fd8e263c788`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:47.281569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-47602e3a-54bd-45e3-b3e5-eca48cecb916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:47.318596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb0e53d4b3774959820ac72824b3d4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:47.319157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.319441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.319719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.321542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:47.350600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1105/2178] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:47.355329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0cd03-d241-4746-8760-259fc94bfa68 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.355329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0cd03-d241-4746-8760-259fc94bfa68 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.355329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0cd03-d241-4746-8760-259fc94bfa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.375382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-36d0cd03-d241-4746-8760-259fc94bfa68 None None] MFA Rules not processed for user `ceed468da0c844b688f5f91768b0dd96`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:47.376731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-36d0cd03-d241-4746-8760-259fc94bfa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:47.389167 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:47.390697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:47.400026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1074/2179] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:47.400960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:47.402391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:47.450499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1106/2180] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:47.480552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320678c-0820-48bf-8c09-4ea4e1be9477 service glance] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:47.484478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef716e6-e24a-453c-aa75-d11430d77188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:47.485168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef716e6-e24a-453c-aa75-d11430d77188 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.485451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef716e6-e24a-453c-aa75-d11430d77188 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.485724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef716e6-e24a-453c-aa75-d11430d77188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.498202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef716e6-e24a-453c-aa75-d11430d77188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:47.499753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef716e6-e24a-453c-aa75-d11430d77188 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:47.517006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1075/2181] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:47.534291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0cd03-d241-4746-8760-259fc94bfa68 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:47.658295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0cd03-d241-4746-8760-259fc94bfa68 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:47.666134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:47.667883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.668338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.668772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] PATH_INFO: `/v3/users/b27d09f23f374e5c93b84fd8e263c788` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.670311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b27d09f23f374e5c93b84fd8e263c788)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:47.672505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:47.727401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:47.759679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b27d09f23f374e5c93b84fd8e263c788'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:47.787866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b27d09f23f374e5c93b84fd8e263c788'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:47.800813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b27d09f23f374e5c93b84fd8e263c788'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:47.801799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b27d09f23f374e5c93b84fd8e263c788'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:47.802414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b27d09f23f374e5c93b84fd8e263c788'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:47.820803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1107/2182] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:47 2022] DELETE /identity/v3/users/b27d09f23f374e5c93b84fd8e263c788 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:47.826083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1971d7-5a94-4a76-a4e8-bd797bf3981f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:47.826646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1971d7-5a94-4a76-a4e8-bd797bf3981f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:47.826646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1971d7-5a94-4a76-a4e8-bd797bf3981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:47.845845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4d1971d7-5a94-4a76-a4e8-bd797bf3981f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:47.847059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4d1971d7-5a94-4a76-a4e8-bd797bf3981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:47.906085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:47.937521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1076/2183] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:47.956927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:47.958641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:48.052556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bbcb7-298d-400e-99cd-2e26251b5491 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:48.056410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d11989-96a5-4822-ba16-28a23ceae4c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:48.057279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11989-96a5-4822-ba16-28a23ceae4c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:48.057375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11989-96a5-4822-ba16-28a23ceae4c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:48.057737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11989-96a5-4822-ba16-28a23ceae4c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:48.072328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d11989-96a5-4822-ba16-28a23ceae4c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:48.078816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d11989-96a5-4822-ba16-28a23ceae4c0 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:48.092909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1108/2184] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:48.539516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1971d7-5a94-4a76-a4e8-bd797bf3981f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:48.541022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4d1971d7-5a94-4a76-a4e8-bd797bf3981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:48.554531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1971d7-5a94-4a76-a4e8-bd797bf3981f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:48.561462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:48.562317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:48.562654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:48.562934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] PATH_INFO: `/v3/projects/97995b2cf66948f4984ff7804e90a431` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:48.564150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97995b2cf66948f4984ff7804e90a431)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:48.566178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:48.628572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97995b2cf66948f4984ff7804e90a431', 'request_id': 'req-c9a06bc7-4aa2-4768-acf5-415e256e1135'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:48.636228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97995b2cf66948f4984ff7804e90a431', 'request_id': 'req-c9a06bc7-4aa2-4768-acf5-415e256e1135'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:48.646019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97995b2cf66948f4984ff7804e90a431', 'request_id': 'req-c9a06bc7-4aa2-4768-acf5-415e256e1135'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:48.660771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] The token cache is being invalidate because project 97995b2cf66948f4984ff7804e90a431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:48.661230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:48.662023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c9a06bc7-4aa2-4768-acf5-415e256e1135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:48.663747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1077/2185] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:48 2022] DELETE /identity/v3/projects/97995b2cf66948f4984ff7804e90a431 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:48.668993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453acca-efc3-4ae9-87e8-52b603eb5f76 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:48.669492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453acca-efc3-4ae9-87e8-52b603eb5f76 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:48.669601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453acca-efc3-4ae9-87e8-52b603eb5f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:48.692791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-5453acca-efc3-4ae9-87e8-52b603eb5f76 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:48.694182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5453acca-efc3-4ae9-87e8-52b603eb5f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:48.764270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1109/2186] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:48.766390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955ef0e-326b-4df2-aa1c-74eb2f83d272 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:48.766678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955ef0e-326b-4df2-aa1c-74eb2f83d272 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:48.766956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955ef0e-326b-4df2-aa1c-74eb2f83d272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:48.786505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d955ef0e-326b-4df2-aa1c-74eb2f83d272 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:48.787722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d955ef0e-326b-4df2-aa1c-74eb2f83d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:48.798621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1078/2187] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:48.802678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453acca-efc3-4ae9-87e8-52b603eb5f76 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:48.818679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453acca-efc3-4ae9-87e8-52b603eb5f76 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:48.823461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c022ff-019f-446a-b84c-02f74632cb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:48.824328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c022ff-019f-446a-b84c-02f74632cb53 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:48.824616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c022ff-019f-446a-b84c-02f74632cb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:48.824926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c022ff-019f-446a-b84c-02f74632cb53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:48.825669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c022ff-019f-446a-b84c-02f74632cb53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:48.825985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c022ff-019f-446a-b84c-02f74632cb53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:48.827593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c022ff-019f-446a-b84c-02f74632cb53 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:48.839716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1110/2188] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:48.844067 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955ef0e-326b-4df2-aa1c-74eb2f83d272 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:48.857798 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955ef0e-326b-4df2-aa1c-74eb2f83d272 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:48.860935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c113e-36af-4811-9bda-6deaa63a4728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:48.861645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c113e-36af-4811-9bda-6deaa63a4728 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:48.861936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c113e-36af-4811-9bda-6deaa63a4728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:48.862211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c113e-36af-4811-9bda-6deaa63a4728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:48.863225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c113e-36af-4811-9bda-6deaa63a4728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:48.866084 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c113e-36af-4811-9bda-6deaa63a4728 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:48.902720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1079/2189] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:48 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:48.906381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c022ff-019f-446a-b84c-02f74632cb53 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:48.916850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c022ff-019f-446a-b84c-02f74632cb53 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:48.965647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:48.997004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21840d64-85d1-4cfd-86e0-7423a15cfcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:48.997672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21840d64-85d1-4cfd-86e0-7423a15cfcb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:48.998007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21840d64-85d1-4cfd-86e0-7423a15cfcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:48.998630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21840d64-85d1-4cfd-86e0-7423a15cfcb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:48.999031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21840d64-85d1-4cfd-86e0-7423a15cfcb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.001174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21840d64-85d1-4cfd-86e0-7423a15cfcb7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.028622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1111/2190] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:48 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:49.032668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c113e-36af-4811-9bda-6deaa63a4728 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.044337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c113e-36af-4811-9bda-6deaa63a4728 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.049328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104caf4-1b92-4f1a-adcf-b9f14b79e433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.049997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104caf4-1b92-4f1a-adcf-b9f14b79e433 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.050280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104caf4-1b92-4f1a-adcf-b9f14b79e433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.050552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104caf4-1b92-4f1a-adcf-b9f14b79e433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.051304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104caf4-1b92-4f1a-adcf-b9f14b79e433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.052976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104caf4-1b92-4f1a-adcf-b9f14b79e433 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.064607 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1080/2191] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:49.068832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21840d64-85d1-4cfd-86e0-7423a15cfcb7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.079518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21840d64-85d1-4cfd-86e0-7423a15cfcb7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.084287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a2069-ae7e-448f-8441-dbc497dcf958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.084997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a2069-ae7e-448f-8441-dbc497dcf958 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.085283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a2069-ae7e-448f-8441-dbc497dcf958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.085585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a2069-ae7e-448f-8441-dbc497dcf958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.086316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a2069-ae7e-448f-8441-dbc497dcf958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.087970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a2069-ae7e-448f-8441-dbc497dcf958 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.098783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1112/2192] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:49.103182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104caf4-1b92-4f1a-adcf-b9f14b79e433 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.114296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104caf4-1b92-4f1a-adcf-b9f14b79e433 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.121091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8335d-8e6b-4093-95b5-4c5fbfd8a32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.122661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8335d-8e6b-4093-95b5-4c5fbfd8a32b admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.122973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8335d-8e6b-4093-95b5-4c5fbfd8a32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.123251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8335d-8e6b-4093-95b5-4c5fbfd8a32b admin admin] PATH_INFO: `/v3/projects/812a159de22043d8a66ebed115bf1670/users/19f54dcd80914009955d181cc926bf16/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.140029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8335d-8e6b-4093-95b5-4c5fbfd8a32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=812a159de22043d8a66ebed115bf1670, user_id=19f54dcd80914009955d181cc926bf16, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.142726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8335d-8e6b-4093-95b5-4c5fbfd8a32b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.160720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1081/2193] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:49 2022] PUT /identity/v3/projects/812a159de22043d8a66ebed115bf1670/users/19f54dcd80914009955d181cc926bf16/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:49.171853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae61e91-8af8-463e-b56f-d4d548294c6d None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.172166 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae61e91-8af8-463e-b56f-d4d548294c6d None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.172477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae61e91-8af8-463e-b56f-d4d548294c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.201187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-7ae61e91-8af8-463e-b56f-d4d548294c6d None None] MFA Rules not processed for user `19f54dcd80914009955d181cc926bf16`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:49.202398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7ae61e91-8af8-463e-b56f-d4d548294c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:49.270151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1113/2194] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:49.274964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8335d-8e6b-4093-95b5-4c5fbfd8a32b admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.342777 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8335d-8e6b-4093-95b5-4c5fbfd8a32b admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.347471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1def04a-8906-473a-80e1-d87a87c3d8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.348069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1def04a-8906-473a-80e1-d87a87c3d8fa admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.348354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1def04a-8906-473a-80e1-d87a87c3d8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.348621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1def04a-8906-473a-80e1-d87a87c3d8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.349442 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1def04a-8906-473a-80e1-d87a87c3d8fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.351299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1def04a-8906-473a-80e1-d87a87c3d8fa admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.361974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:49.380190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1082/2195] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:49 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:49.393321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae61e91-8af8-463e-b56f-d4d548294c6d None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.404307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae61e91-8af8-463e-b56f-d4d548294c6d None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.464592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11434f48-0f30-40d3-b56e-2e53c5234b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.465316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11434f48-0f30-40d3-b56e-2e53c5234b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.465604 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11434f48-0f30-40d3-b56e-2e53c5234b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.465884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11434f48-0f30-40d3-b56e-2e53c5234b75 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.466578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11434f48-0f30-40d3-b56e-2e53c5234b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.468573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11434f48-0f30-40d3-b56e-2e53c5234b75 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.498433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1114/2196] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:49 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:49.499252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1def04a-8906-473a-80e1-d87a87c3d8fa admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.511388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1def04a-8906-473a-80e1-d87a87c3d8fa admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.520234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4466e49-0ead-42b2-82b8-5b1eb41dfc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.521744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4466e49-0ead-42b2-82b8-5b1eb41dfc16 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.522036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4466e49-0ead-42b2-82b8-5b1eb41dfc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.522316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4466e49-0ead-42b2-82b8-5b1eb41dfc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.523095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4466e49-0ead-42b2-82b8-5b1eb41dfc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.524861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4466e49-0ead-42b2-82b8-5b1eb41dfc16 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.536661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1083/2197] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:49.540817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11434f48-0f30-40d3-b56e-2e53c5234b75 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.551993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11434f48-0f30-40d3-b56e-2e53c5234b75 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.557679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747a5c9-20c3-44ac-b0fa-84262408861f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.559305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747a5c9-20c3-44ac-b0fa-84262408861f admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.559643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747a5c9-20c3-44ac-b0fa-84262408861f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.559860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747a5c9-20c3-44ac-b0fa-84262408861f admin admin] PATH_INFO: `/v3/projects/c6e0cf0a68d04466b88859cf05434f3c/users/742acaa1c8ad420593aaf57006f6274d/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.577447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747a5c9-20c3-44ac-b0fa-84262408861f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6e0cf0a68d04466b88859cf05434f3c, user_id=742acaa1c8ad420593aaf57006f6274d, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.580304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747a5c9-20c3-44ac-b0fa-84262408861f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.602279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1115/2198] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:49 2022] PUT /identity/v3/projects/c6e0cf0a68d04466b88859cf05434f3c/users/742acaa1c8ad420593aaf57006f6274d/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:49.607186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4466e49-0ead-42b2-82b8-5b1eb41dfc16 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.622536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4466e49-0ead-42b2-82b8-5b1eb41dfc16 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.708520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbbfe4-95c2-4cce-bad7-a6882872e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.709927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbbfe4-95c2-4cce-bad7-a6882872e0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.710309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbbfe4-95c2-4cce-bad7-a6882872e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.710675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbbfe4-95c2-4cce-bad7-a6882872e0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.711495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbbfe4-95c2-4cce-bad7-a6882872e0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.718275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbbfe4-95c2-4cce-bad7-a6882872e0c4 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.732374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1084/2199] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:49.733866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747a5c9-20c3-44ac-b0fa-84262408861f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.746193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747a5c9-20c3-44ac-b0fa-84262408861f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.751433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314318-452a-46ef-86af-586079b09d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.752326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314318-452a-46ef-86af-586079b09d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.752606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314318-452a-46ef-86af-586079b09d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.752930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314318-452a-46ef-86af-586079b09d2f admin admin] PATH_INFO: `/v3/projects/c6e0cf0a68d04466b88859cf05434f3c/users/742acaa1c8ad420593aaf57006f6274d/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.754919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10314318-452a-46ef-86af-586079b09d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6e0cf0a68d04466b88859cf05434f3c, user_id=742acaa1c8ad420593aaf57006f6274d, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:49.757655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10314318-452a-46ef-86af-586079b09d2f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:49.764667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:49.771313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1116/2200] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:49 2022] PUT /identity/v3/projects/c6e0cf0a68d04466b88859cf05434f3c/users/742acaa1c8ad420593aaf57006f6274d/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:49.782360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a3517-4581-4ccc-9db2-4b90a781ab33 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.782645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a3517-4581-4ccc-9db2-4b90a781ab33 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.782953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a3517-4581-4ccc-9db2-4b90a781ab33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:49.811121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-cf0a3517-4581-4ccc-9db2-4b90a781ab33 None None] MFA Rules not processed for user `742acaa1c8ad420593aaf57006f6274d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:49.812334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-cf0a3517-4581-4ccc-9db2-4b90a781ab33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:49.893325 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1085/2201] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:49.906902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314318-452a-46ef-86af-586079b09d2f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:49.908764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-10314318-452a-46ef-86af-586079b09d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:49.976781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314318-452a-46ef-86af-586079b09d2f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:49.983653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb89343-3380-4961-a482-ac8a891b3920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:49.984318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb89343-3380-4961-a482-ac8a891b3920 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:49.984600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb89343-3380-4961-a482-ac8a891b3920 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:49.984890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb89343-3380-4961-a482-ac8a891b3920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:50.055913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb89343-3380-4961-a482-ac8a891b3920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:50.057919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb89343-3380-4961-a482-ac8a891b3920 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:50.073994 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1117/2202] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:52.779087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a3517-4581-4ccc-9db2-4b90a781ab33 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:52.877447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a3517-4581-4ccc-9db2-4b90a781ab33 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:52.883794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:52.885467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:52.885987 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:52.886457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] PATH_INFO: `/v3/users/19f54dcd80914009955d181cc926bf16` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:52.888746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f54dcd80914009955d181cc926bf16)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:52.891954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:52.896014 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:52.969864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f54dcd80914009955d181cc926bf16'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:52.970515 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f54dcd80914009955d181cc926bf16'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:52.979114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f54dcd80914009955d181cc926bf16'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:52.980650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f54dcd80914009955d181cc926bf16'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:52.995254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f54dcd80914009955d181cc926bf16'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.030586 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1086/2203] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:52 2022] DELETE /identity/v3/users/19f54dcd80914009955d181cc926bf16 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:53.032318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5832c-d99b-4649-811b-1510126ea8ef None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:53.032788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5832c-d99b-4649-811b-1510126ea8ef None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:53.032919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5832c-d99b-4649-811b-1510126ea8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:53.061411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1db5832c-d99b-4649-811b-1510126ea8ef None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:53.063575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1db5832c-d99b-4649-811b-1510126ea8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:53.160227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1118/2204] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:53.173819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:53.175344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:53.255350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b3f4b-2a50-4d03-851f-46ca6d989da1 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:53.263984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f26f8-89c0-4891-930c-c2f2abd34193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:53.265166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f26f8-89c0-4891-930c-c2f2abd34193 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:53.265555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f26f8-89c0-4891-930c-c2f2abd34193 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:53.266350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f26f8-89c0-4891-930c-c2f2abd34193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:53.285697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f26f8-89c0-4891-930c-c2f2abd34193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:53.290571 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f26f8-89c0-4891-930c-c2f2abd34193 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:53.317425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1087/2205] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:31:53.649781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5832c-d99b-4649-811b-1510126ea8ef None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:53.651190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1db5832c-d99b-4649-811b-1510126ea8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:53.663838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5832c-d99b-4649-811b-1510126ea8ef None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:53.671509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:53.671895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:53.672291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:53.672656 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] PATH_INFO: `/v3/projects/812a159de22043d8a66ebed115bf1670` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:53.674412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=812a159de22043d8a66ebed115bf1670)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:53.676726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:53.725484 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:53.726717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '812a159de22043d8a66ebed115bf1670', 'request_id': 'req-4ff1a698-030a-473b-9c69-b07cae37d124'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.734780 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '812a159de22043d8a66ebed115bf1670', 'request_id': 'req-4ff1a698-030a-473b-9c69-b07cae37d124'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.740476 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '812a159de22043d8a66ebed115bf1670', 'request_id': 'req-4ff1a698-030a-473b-9c69-b07cae37d124'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.752774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] The token cache is being invalidate because project 812a159de22043d8a66ebed115bf1670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:53.753096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.753771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.755309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1119/2206] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:53 2022] DELETE /identity/v3/projects/812a159de22043d8a66ebed115bf1670 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:53.759495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f26f8-89c0-4891-930c-c2f2abd34193 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:53.769551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f26f8-89c0-4891-930c-c2f2abd34193 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:53.829779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:53.830661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:53.830950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:53.831228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] PATH_INFO: `/v3/users/742acaa1c8ad420593aaf57006f6274d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:53.832471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=742acaa1c8ad420593aaf57006f6274d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:53.834464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:53.862489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:53.881835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742acaa1c8ad420593aaf57006f6274d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.882485 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742acaa1c8ad420593aaf57006f6274d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.890678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742acaa1c8ad420593aaf57006f6274d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.891331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742acaa1c8ad420593aaf57006f6274d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.905375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fbb3bda5-8f67-4491-886a-58d27e4ed9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742acaa1c8ad420593aaf57006f6274d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:53.919034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1088/2207] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:31:53 2022] DELETE /identity/v3/users/742acaa1c8ad420593aaf57006f6274d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:54.217354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:54.219822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:54.235472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1a698-030a-473b-9c69-b07cae37d124 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:54.317759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:54.318722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:54.319037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:54.319339 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] PATH_INFO: `/v3/projects/c6e0cf0a68d04466b88859cf05434f3c` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:54.321185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6e0cf0a68d04466b88859cf05434f3c)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:54.324064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:54.385401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6e0cf0a68d04466b88859cf05434f3c', 'request_id': 'req-615e1be2-4a68-4c51-8b15-0ca000fb194d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:54.393420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6e0cf0a68d04466b88859cf05434f3c', 'request_id': 'req-615e1be2-4a68-4c51-8b15-0ca000fb194d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:54.400650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6e0cf0a68d04466b88859cf05434f3c', 'request_id': 'req-615e1be2-4a68-4c51-8b15-0ca000fb194d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:54.419151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] The token cache is being invalidate because project c6e0cf0a68d04466b88859cf05434f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:31:54.419428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:54.420511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-615e1be2-4a68-4c51-8b15-0ca000fb194d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:31:54.422654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1120/2208] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:31:54 2022] DELETE /identity/v3/projects/c6e0cf0a68d04466b88859cf05434f3c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:54.427706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e080d-24e4-4244-b8b0-fca20a5fec9a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:54.428038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e080d-24e4-4244-b8b0-fca20a5fec9a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:54.428310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e080d-24e4-4244-b8b0-fca20a5fec9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:54.492813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-672e080d-24e4-4244-b8b0-fca20a5fec9a None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:54.494478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-672e080d-24e4-4244-b8b0-fca20a5fec9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:54.609584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1089/2209] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:54.617071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89b50-a140-4671-bae3-7f3d35ce2284 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:54.617360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89b50-a140-4671-bae3-7f3d35ce2284 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:54.617639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89b50-a140-4671-bae3-7f3d35ce2284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:54.647925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-67c89b50-a140-4671-bae3-7f3d35ce2284 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:54.649160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-67c89b50-a140-4671-bae3-7f3d35ce2284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:54.660467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1121/2210] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:54.661871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e080d-24e4-4244-b8b0-fca20a5fec9a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:54.677261 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e080d-24e4-4244-b8b0-fca20a5fec9a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:54.684991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec1615-1a9a-4520-a3b3-f1294a9b158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:54.684991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec1615-1a9a-4520-a3b3-f1294a9b158b admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:54.684991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec1615-1a9a-4520-a3b3-f1294a9b158b admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:54.684991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec1615-1a9a-4520-a3b3-f1294a9b158b admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:54.685459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec1615-1a9a-4520-a3b3-f1294a9b158b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:31:54.685620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec1615-1a9a-4520-a3b3-f1294a9b158b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:54.687350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec1615-1a9a-4520-a3b3-f1294a9b158b admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:54.700102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1090/2211] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:31:54.705405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c89b50-a140-4671-bae3-7f3d35ce2284 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:54.717218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c89b50-a140-4671-bae3-7f3d35ce2284 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:54.722829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6ea97-803f-48c4-9f59-febcbe55b4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:54.723819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6ea97-803f-48c4-9f59-febcbe55b4bf admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:54.724202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6ea97-803f-48c4-9f59-febcbe55b4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:54.724579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6ea97-803f-48c4-9f59-febcbe55b4bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:54.725417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6ea97-803f-48c4-9f59-febcbe55b4bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:54.727530 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6ea97-803f-48c4-9f59-febcbe55b4bf admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:54.767278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1122/2212] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:54 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:54.773647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec1615-1a9a-4520-a3b3-f1294a9b158b admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:54.785658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec1615-1a9a-4520-a3b3-f1294a9b158b admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:54.856993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d58b1-b42c-4aaa-80a3-56173feaf6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:54.858083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d58b1-b42c-4aaa-80a3-56173feaf6de admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:54.858458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d58b1-b42c-4aaa-80a3-56173feaf6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:54.858772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d58b1-b42c-4aaa-80a3-56173feaf6de admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:54.859683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d58b1-b42c-4aaa-80a3-56173feaf6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:54.861912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d58b1-b42c-4aaa-80a3-56173feaf6de admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:54.885252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1091/2213] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:54 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:31:54.890037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6ea97-803f-48c4-9f59-febcbe55b4bf admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:54.902051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6ea97-803f-48c4-9f59-febcbe55b4bf admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:54.909787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634711-7aed-4aff-ba6a-ad04b980bf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:54.910540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634711-7aed-4aff-ba6a-ad04b980bf44 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:54.910823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634711-7aed-4aff-ba6a-ad04b980bf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:54.911099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634711-7aed-4aff-ba6a-ad04b980bf44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:54.911841 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634711-7aed-4aff-ba6a-ad04b980bf44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:54.913733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634711-7aed-4aff-ba6a-ad04b980bf44 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:54.930055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1123/2214] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:54.931228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d58b1-b42c-4aaa-80a3-56173feaf6de admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:54.943777 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d58b1-b42c-4aaa-80a3-56173feaf6de admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:54.948644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6491632-853a-456a-9782-9a97b182291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:54.949517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6491632-853a-456a-9782-9a97b182291f admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:54.949874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6491632-853a-456a-9782-9a97b182291f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:54.950165 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6491632-853a-456a-9782-9a97b182291f admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:54.950902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6491632-853a-456a-9782-9a97b182291f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:54.952529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6491632-853a-456a-9782-9a97b182291f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:54.958079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:54.965447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1092/2215] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:54.969809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634711-7aed-4aff-ba6a-ad04b980bf44 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:54.981402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634711-7aed-4aff-ba6a-ad04b980bf44 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:54.986011 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bda6b-8608-4632-b2e1-da3f7e381c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:54.986896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bda6b-8608-4632-b2e1-da3f7e381c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:54.987175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bda6b-8608-4632-b2e1-da3f7e381c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:54.987449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bda6b-8608-4632-b2e1-da3f7e381c49 admin admin] PATH_INFO: `/v3/projects/15e1c2f2d0194933b32081b460a0a9fc/users/eada72f3c9d74d228050a35737a6fe5d/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.003251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bda6b-8608-4632-b2e1-da3f7e381c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e1c2f2d0194933b32081b460a0a9fc, user_id=eada72f3c9d74d228050a35737a6fe5d, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.005950 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bda6b-8608-4632-b2e1-da3f7e381c49 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.020644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1124/2216] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:54 2022] PUT /identity/v3/projects/15e1c2f2d0194933b32081b460a0a9fc/users/eada72f3c9d74d228050a35737a6fe5d/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:55.032827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242743e6-89ec-4378-80ea-21bdaaa968fa None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.033184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242743e6-89ec-4378-80ea-21bdaaa968fa None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.033497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242743e6-89ec-4378-80ea-21bdaaa968fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.062521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-242743e6-89ec-4378-80ea-21bdaaa968fa None None] MFA Rules not processed for user `eada72f3c9d74d228050a35737a6fe5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:55.063863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-242743e6-89ec-4378-80ea-21bdaaa968fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:55.133670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1093/2217] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:31:55.139273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bda6b-8608-4632-b2e1-da3f7e381c49 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.143693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:55.210761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bda6b-8608-4632-b2e1-da3f7e381c49 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.218979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348bee2-d0c7-465f-abae-0d94757a4efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:55.219674 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348bee2-d0c7-465f-abae-0d94757a4efc admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.219962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348bee2-d0c7-465f-abae-0d94757a4efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.220252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348bee2-d0c7-465f-abae-0d94757a4efc admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.220984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348bee2-d0c7-465f-abae-0d94757a4efc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.222948 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348bee2-d0c7-465f-abae-0d94757a4efc admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.251162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1125/2218] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:55 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:55.255511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242743e6-89ec-4378-80ea-21bdaaa968fa None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.270779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242743e6-89ec-4378-80ea-21bdaaa968fa None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.332984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf877b3-8b61-4a2e-af2a-73a69ab56eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:55.333769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf877b3-8b61-4a2e-af2a-73a69ab56eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.334122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf877b3-8b61-4a2e-af2a-73a69ab56eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.334468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf877b3-8b61-4a2e-af2a-73a69ab56eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.335244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf877b3-8b61-4a2e-af2a-73a69ab56eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.337978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf877b3-8b61-4a2e-af2a-73a69ab56eb9 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.365967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1094/2219] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:55 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:55.369920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348bee2-d0c7-465f-abae-0d94757a4efc admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.382943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348bee2-d0c7-465f-abae-0d94757a4efc admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.390020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dae21f-f1a0-44bc-af40-2a30de54d3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:55.390716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dae21f-f1a0-44bc-af40-2a30de54d3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.391062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dae21f-f1a0-44bc-af40-2a30de54d3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.391519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dae21f-f1a0-44bc-af40-2a30de54d3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.392456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dae21f-f1a0-44bc-af40-2a30de54d3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.394229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dae21f-f1a0-44bc-af40-2a30de54d3f7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.410985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1126/2220] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:55.412330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf877b3-8b61-4a2e-af2a-73a69ab56eb9 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.423357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf877b3-8b61-4a2e-af2a-73a69ab56eb9 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.429081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac645bf8-9fba-451a-814d-c24fc02492fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:55.430009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645bf8-9fba-451a-814d-c24fc02492fe admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.430285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645bf8-9fba-451a-814d-c24fc02492fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.430683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645bf8-9fba-451a-814d-c24fc02492fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.431566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac645bf8-9fba-451a-814d-c24fc02492fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.433990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac645bf8-9fba-451a-814d-c24fc02492fe admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.444900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1095/2221] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:55.449653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dae21f-f1a0-44bc-af40-2a30de54d3f7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.459960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dae21f-f1a0-44bc-af40-2a30de54d3f7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.464535 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93ac40-f996-4978-b73d-b01a485056e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:55.465447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93ac40-f996-4978-b73d-b01a485056e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.465730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93ac40-f996-4978-b73d-b01a485056e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.466006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93ac40-f996-4978-b73d-b01a485056e8 admin admin] PATH_INFO: `/v3/projects/aafa6ec5fac34ed383861cb53430c2c4/users/2b4e838c5de148da805577282aa74f3d/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.480618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93ac40-f996-4978-b73d-b01a485056e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aafa6ec5fac34ed383861cb53430c2c4, user_id=2b4e838c5de148da805577282aa74f3d, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.483445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93ac40-f996-4978-b73d-b01a485056e8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.499961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1127/2222] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:55 2022] PUT /identity/v3/projects/aafa6ec5fac34ed383861cb53430c2c4/users/2b4e838c5de148da805577282aa74f3d/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:55.510527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d6541-9c9c-41c3-a52a-3c84e051460f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.510850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d6541-9c9c-41c3-a52a-3c84e051460f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.511628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d6541-9c9c-41c3-a52a-3c84e051460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.544795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e40d6541-9c9c-41c3-a52a-3c84e051460f None None] MFA Rules not processed for user `2b4e838c5de148da805577282aa74f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:55.546042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e40d6541-9c9c-41c3-a52a-3c84e051460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:55.607930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:55.614201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1096/2223] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:55.618334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93ac40-f996-4978-b73d-b01a485056e8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.687426 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93ac40-f996-4978-b73d-b01a485056e8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.694367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c620d2-42f4-46fa-a919-19585c4104ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:55.695483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c620d2-42f4-46fa-a919-19585c4104ea admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.695965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c620d2-42f4-46fa-a919-19585c4104ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.696329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c620d2-42f4-46fa-a919-19585c4104ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.697148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c620d2-42f4-46fa-a919-19585c4104ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.699254 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c620d2-42f4-46fa-a919-19585c4104ea admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.730367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1128/2224] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:31:55 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:55.731601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d6541-9c9c-41c3-a52a-3c84e051460f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.748847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d6541-9c9c-41c3-a52a-3c84e051460f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.808763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838c618-d276-4865-8c94-641752a81c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:55.809477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838c618-d276-4865-8c94-641752a81c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.809763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838c618-d276-4865-8c94-641752a81c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.810041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838c618-d276-4865-8c94-641752a81c78 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.810737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838c618-d276-4865-8c94-641752a81c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.812687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838c618-d276-4865-8c94-641752a81c78 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.835320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1097/2225] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:31:55 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:31:55.839815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c620d2-42f4-46fa-a919-19585c4104ea admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.849997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c620d2-42f4-46fa-a919-19585c4104ea admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.855760 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41f404-a600-4b7e-aa30-0e2de69cfa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:55.856732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41f404-a600-4b7e-aa30-0e2de69cfa4c admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.857120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41f404-a600-4b7e-aa30-0e2de69cfa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.857511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41f404-a600-4b7e-aa30-0e2de69cfa4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.858440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41f404-a600-4b7e-aa30-0e2de69cfa4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.860453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41f404-a600-4b7e-aa30-0e2de69cfa4c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.872205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1129/2226] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:31:55.877386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838c618-d276-4865-8c94-641752a81c78 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.888460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838c618-d276-4865-8c94-641752a81c78 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.893228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d601de-18cb-4e54-8e6d-61b40b01fbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:55.894048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d601de-18cb-4e54-8e6d-61b40b01fbae admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:55.894439 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d601de-18cb-4e54-8e6d-61b40b01fbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:55.894724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d601de-18cb-4e54-8e6d-61b40b01fbae admin admin] PATH_INFO: `/v3/projects/551ad2d8402c48d1a319e4c23bf487e3/users/87ca8b592e9f4e9680a64ec7bf4ffe59/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:55.910825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d601de-18cb-4e54-8e6d-61b40b01fbae admin admin] RBAC: Authorizing `identity:create_grant(project_id=551ad2d8402c48d1a319e4c23bf487e3, user_id=87ca8b592e9f4e9680a64ec7bf4ffe59, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:55.913568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d601de-18cb-4e54-8e6d-61b40b01fbae admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:55.929489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1098/2227] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:55 2022] PUT /identity/v3/projects/551ad2d8402c48d1a319e4c23bf487e3/users/87ca8b592e9f4e9680a64ec7bf4ffe59/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:31:55.933810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41f404-a600-4b7e-aa30-0e2de69cfa4c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:55.947733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41f404-a600-4b7e-aa30-0e2de69cfa4c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:55.978889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:56.014715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f2966-219d-4668-bddf-7b344f15bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:56.015585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f2966-219d-4668-bddf-7b344f15bcdd admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:56.015867 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f2966-219d-4668-bddf-7b344f15bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:56.016162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f2966-219d-4668-bddf-7b344f15bcdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:56.016943 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f2966-219d-4668-bddf-7b344f15bcdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:56.018606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f2966-219d-4668-bddf-7b344f15bcdd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:56.030537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1130/2228] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:31:56.034740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d601de-18cb-4e54-8e6d-61b40b01fbae admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:56.046620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d601de-18cb-4e54-8e6d-61b40b01fbae admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:56.051248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899a350-8922-468f-8348-17dd54f5bd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:56.052123 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899a350-8922-468f-8348-17dd54f5bd40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:56.052406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899a350-8922-468f-8348-17dd54f5bd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:56.052676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899a350-8922-468f-8348-17dd54f5bd40 admin admin] PATH_INFO: `/v3/projects/551ad2d8402c48d1a319e4c23bf487e3/users/87ca8b592e9f4e9680a64ec7bf4ffe59/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:56.054651 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899a350-8922-468f-8348-17dd54f5bd40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=551ad2d8402c48d1a319e4c23bf487e3, user_id=87ca8b592e9f4e9680a64ec7bf4ffe59, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:56.057488 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899a350-8922-468f-8348-17dd54f5bd40 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:56.067919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1099/2229] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:31:56 2022] PUT /identity/v3/projects/551ad2d8402c48d1a319e4c23bf487e3/users/87ca8b592e9f4e9680a64ec7bf4ffe59/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:31:56.079726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43da9f-365d-491a-8b3f-e437cd8761cd None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:56.079994 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43da9f-365d-491a-8b3f-e437cd8761cd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:56.080273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43da9f-365d-491a-8b3f-e437cd8761cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:56.107006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-fb43da9f-365d-491a-8b3f-e437cd8761cd None None] MFA Rules not processed for user `87ca8b592e9f4e9680a64ec7bf4ffe59`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:31:56.108195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fb43da9f-365d-491a-8b3f-e437cd8761cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:56.178633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1131/2230] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:31:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:31:56.190065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899a350-8922-468f-8348-17dd54f5bd40 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:56.191747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1899a350-8922-468f-8348-17dd54f5bd40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:56.252468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899a350-8922-468f-8348-17dd54f5bd40 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:56.256504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4220ad-fbcf-4aae-8d93-7924cf810dd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:56.257128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4220ad-fbcf-4aae-8d93-7924cf810dd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:56.257412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4220ad-fbcf-4aae-8d93-7924cf810dd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:56.257702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4220ad-fbcf-4aae-8d93-7924cf810dd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:56.328136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4220ad-fbcf-4aae-8d93-7924cf810dd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:56.329664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4220ad-fbcf-4aae-8d93-7924cf810dd2 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:56.338606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1100/2231] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:57.923191 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43da9f-365d-491a-8b3f-e437cd8761cd None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:57.924716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fb43da9f-365d-491a-8b3f-e437cd8761cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:31:57.937211 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43da9f-365d-491a-8b3f-e437cd8761cd None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:57.940743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7daca-99da-44bd-aa4f-d3b67862a6cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:57.941437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7daca-99da-44bd-aa4f-d3b67862a6cc service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:57.941720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7daca-99da-44bd-aa4f-d3b67862a6cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:57.941996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7daca-99da-44bd-aa4f-d3b67862a6cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:58.001527 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7daca-99da-44bd-aa4f-d3b67862a6cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:58.002952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7daca-99da-44bd-aa4f-d3b67862a6cc service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:58.012848 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1132/2232] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:31:58.593787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4220ad-fbcf-4aae-8d93-7924cf810dd2 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:31:58.599045 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:31:58.603974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4220ad-fbcf-4aae-8d93-7924cf810dd2 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:31:58.609897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19a33e-3911-41b7-ad59-fb0465de87d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:31:58.610794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19a33e-3911-41b7-ad59-fb0465de87d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:31:58.611077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19a33e-3911-41b7-ad59-fb0465de87d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:31:58.611350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19a33e-3911-41b7-ad59-fb0465de87d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:31:58.622440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19a33e-3911-41b7-ad59-fb0465de87d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:31:58.624122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19a33e-3911-41b7-ad59-fb0465de87d9 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:31:58.639373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1101/2233] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:31:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:32:00.078154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f418-03b7-417c-89da-e485832b26be None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:00.078667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f418-03b7-417c-89da-e485832b26be None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:00.078758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f418-03b7-417c-89da-e485832b26be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:00.109133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-db52f418-03b7-417c-89da-e485832b26be None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:00.110383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-db52f418-03b7-417c-89da-e485832b26be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:00.151702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:00.176003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1133/2234] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:00.193193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19a33e-3911-41b7-ad59-fb0465de87d9 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:00.194676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0b19a33e-3911-41b7-ad59-fb0465de87d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:00.271737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19a33e-3911-41b7-ad59-fb0465de87d9 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:00.277392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c07837-30ea-434e-aafe-1db8083b56f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:00.278088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c07837-30ea-434e-aafe-1db8083b56f7 service swift] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:00.278376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c07837-30ea-434e-aafe-1db8083b56f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:00.278676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c07837-30ea-434e-aafe-1db8083b56f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:00.286780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c07837-30ea-434e-aafe-1db8083b56f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:00.288383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c07837-30ea-434e-aafe-1db8083b56f7 service swift] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:00.297138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1102/2235] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:32:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:32:02.691750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f418-03b7-417c-89da-e485832b26be None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:02.806271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f418-03b7-417c-89da-e485832b26be None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:02.812362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:02.813595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:02.813887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:02.814163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] PATH_INFO: `/v3/users/eada72f3c9d74d228050a35737a6fe5d` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:02.815421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eada72f3c9d74d228050a35737a6fe5d)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:02.819042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:02.965616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eada72f3c9d74d228050a35737a6fe5d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:03.003992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eada72f3c9d74d228050a35737a6fe5d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:03.020812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eada72f3c9d74d228050a35737a6fe5d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:03.021602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eada72f3c9d74d228050a35737a6fe5d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:03.022449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eada72f3c9d74d228050a35737a6fe5d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:03.046762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1134/2236] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:32:02 2022] DELETE /identity/v3/users/eada72f3c9d74d228050a35737a6fe5d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:03.048770 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee13de8-c752-40c6-9839-928029d41b70 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:03.049598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee13de8-c752-40c6-9839-928029d41b70 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:03.049598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee13de8-c752-40c6-9839-928029d41b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:03.071942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-eee13de8-c752-40c6-9839-928029d41b70 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:03.074474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-eee13de8-c752-40c6-9839-928029d41b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:03.164495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1103/2237] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:03.175556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:03.177286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:03.296090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d8e98-d98e-4222-8a32-badd458dce15 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:03.301461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e94cd9-e00d-416f-a5cf-673d707ce0e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:03.302791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e94cd9-e00d-416f-a5cf-673d707ce0e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:03.303100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e94cd9-e00d-416f-a5cf-673d707ce0e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:03.303379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e94cd9-e00d-416f-a5cf-673d707ce0e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:03.321572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e94cd9-e00d-416f-a5cf-673d707ce0e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:03.323932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e94cd9-e00d-416f-a5cf-673d707ce0e9 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:03.339220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1135/2238] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:04.043565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee13de8-c752-40c6-9839-928029d41b70 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:04.045824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-eee13de8-c752-40c6-9839-928029d41b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:04.082131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee13de8-c752-40c6-9839-928029d41b70 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:04.105947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:04.106988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:04.107392 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:04.107782 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] PATH_INFO: `/v3/projects/15e1c2f2d0194933b32081b460a0a9fc` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:04.109755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e1c2f2d0194933b32081b460a0a9fc)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:04.111975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:04.163609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf775e-cdf1-46a7-9d62-708fb5173753 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:04.164189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf775e-cdf1-46a7-9d62-708fb5173753 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:04.164279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf775e-cdf1-46a7-9d62-708fb5173753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:04.174077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:04.207273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-83cf775e-cdf1-46a7-9d62-708fb5173753 None None] MFA Rules not processed for user `06431315dc8040cbbafbe83a3ed8e0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:04.208418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-83cf775e-cdf1-46a7-9d62-708fb5173753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:04.227337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e1c2f2d0194933b32081b460a0a9fc', 'request_id': 'req-aa8e8906-a7bc-4e27-992a-4e5a6261c692'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.256359 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e1c2f2d0194933b32081b460a0a9fc', 'request_id': 'req-aa8e8906-a7bc-4e27-992a-4e5a6261c692'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.269337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e1c2f2d0194933b32081b460a0a9fc', 'request_id': 'req-aa8e8906-a7bc-4e27-992a-4e5a6261c692'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.291917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] The token cache is being invalidate because project 15e1c2f2d0194933b32081b460a0a9fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:32:04.294020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.294836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.299913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1104/2239] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:32:04 2022] DELETE /identity/v3/projects/15e1c2f2d0194933b32081b460a0a9fc => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:04.300883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:04.302696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:04.319452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:04.365980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1136/2240] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:32:04.455599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e8906-a7bc-4e27-992a-4e5a6261c692 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:04.461497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:04.462597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:04.462923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:04.463415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] PATH_INFO: `/v3/users/2b4e838c5de148da805577282aa74f3d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:04.465693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4e838c5de148da805577282aa74f3d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:04.471565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:04.539753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4e838c5de148da805577282aa74f3d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.540425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4e838c5de148da805577282aa74f3d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.549847 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4e838c5de148da805577282aa74f3d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.550885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4e838c5de148da805577282aa74f3d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.570876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4e838c5de148da805577282aa74f3d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:04.592474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1105/2241] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:32:04 2022] DELETE /identity/v3/users/2b4e838c5de148da805577282aa74f3d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:05.158524 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf775e-cdf1-46a7-9d62-708fb5173753 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:05.179647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf775e-cdf1-46a7-9d62-708fb5173753 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:05.305983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:05.307017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:05.307139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:05.307380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] PATH_INFO: `/v3/projects/aafa6ec5fac34ed383861cb53430c2c4` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:05.308750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aafa6ec5fac34ed383861cb53430c2c4)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:05.310931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:05.421040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aafa6ec5fac34ed383861cb53430c2c4', 'request_id': 'req-72e70b39-f0d2-48d3-85a2-3247aed14199'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.429250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aafa6ec5fac34ed383861cb53430c2c4', 'request_id': 'req-72e70b39-f0d2-48d3-85a2-3247aed14199'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.437151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aafa6ec5fac34ed383861cb53430c2c4', 'request_id': 'req-72e70b39-f0d2-48d3-85a2-3247aed14199'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.467817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] The token cache is being invalidate because project aafa6ec5fac34ed383861cb53430c2c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:32:05.468277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.469061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.471051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1137/2242] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:32:05 2022] DELETE /identity/v3/projects/aafa6ec5fac34ed383861cb53430c2c4 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:05.475741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:05.477840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:05.505995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f7c5a-aae9-4da8-8654-05dcf96b2657 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:05.628621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:05.640938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:05.643176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:05.643458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] PATH_INFO: `/v3/users/87ca8b592e9f4e9680a64ec7bf4ffe59` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:05.647319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ca8b592e9f4e9680a64ec7bf4ffe59)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:05.652326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:05.740766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:05.802258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ca8b592e9f4e9680a64ec7bf4ffe59'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.804057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ca8b592e9f4e9680a64ec7bf4ffe59'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.837700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ca8b592e9f4e9680a64ec7bf4ffe59'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.837700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ca8b592e9f4e9680a64ec7bf4ffe59'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.865587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e0eaf12b-b22a-4135-b301-273e885c10a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ca8b592e9f4e9680a64ec7bf4ffe59'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:05.898395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1106/2243] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:32:05 2022] DELETE /identity/v3/users/87ca8b592e9f4e9680a64ec7bf4ffe59 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:06.439144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:06.441653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:06.488099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e70b39-f0d2-48d3-85a2-3247aed14199 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:06.532660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:06.558208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:06.559293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:06.559663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:06.559929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] PATH_INFO: `/v3/projects/551ad2d8402c48d1a319e4c23bf487e3` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:06.561301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=551ad2d8402c48d1a319e4c23bf487e3)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:06.563299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:06.631639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551ad2d8402c48d1a319e4c23bf487e3', 'request_id': 'req-01244e01-fc14-4c3c-a731-3af7485e2539'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:06.640423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551ad2d8402c48d1a319e4c23bf487e3', 'request_id': 'req-01244e01-fc14-4c3c-a731-3af7485e2539'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:06.648021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551ad2d8402c48d1a319e4c23bf487e3', 'request_id': 'req-01244e01-fc14-4c3c-a731-3af7485e2539'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:06.662715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] The token cache is being invalidate because project 551ad2d8402c48d1a319e4c23bf487e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:32:06.663057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:06.663968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-01244e01-fc14-4c3c-a731-3af7485e2539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:06.665945 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1138/2244] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:32:06 2022] DELETE /identity/v3/projects/551ad2d8402c48d1a319e4c23bf487e3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:06.670724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd03a3d-b44b-4d8a-977a-3054322212c3 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:06.671367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd03a3d-b44b-4d8a-977a-3054322212c3 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:06.671367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd03a3d-b44b-4d8a-977a-3054322212c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:06.689955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-3dd03a3d-b44b-4d8a-977a-3054322212c3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:06.691173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3dd03a3d-b44b-4d8a-977a-3054322212c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:06.817790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1107/2245] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:06.826023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89575af-fc18-4c4a-897e-0177da0a77a2 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:06.826619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89575af-fc18-4c4a-897e-0177da0a77a2 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:06.827310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89575af-fc18-4c4a-897e-0177da0a77a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:06.856744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-c89575af-fc18-4c4a-897e-0177da0a77a2 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:06.858173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-c89575af-fc18-4c4a-897e-0177da0a77a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:06.866927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1139/2246] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:06.871477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd03a3d-b44b-4d8a-977a-3054322212c3 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:06.888300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd03a3d-b44b-4d8a-977a-3054322212c3 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:06.902275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74540995-4379-4bf3-87f7-420f81ee8629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:06.903450 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74540995-4379-4bf3-87f7-420f81ee8629 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:06.903968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74540995-4379-4bf3-87f7-420f81ee8629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:06.904458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74540995-4379-4bf3-87f7-420f81ee8629 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:06.906731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74540995-4379-4bf3-87f7-420f81ee8629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:32:06.907262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74540995-4379-4bf3-87f7-420f81ee8629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:06.912399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74540995-4379-4bf3-87f7-420f81ee8629 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:06.926627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1108/2247] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:32:06.930879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89575af-fc18-4c4a-897e-0177da0a77a2 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:06.943538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89575af-fc18-4c4a-897e-0177da0a77a2 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:06.949976 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1422cd-ef24-4919-833a-ef70a75c80e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:06.950725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1422cd-ef24-4919-833a-ef70a75c80e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:06.951007 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1422cd-ef24-4919-833a-ef70a75c80e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:06.951287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1422cd-ef24-4919-833a-ef70a75c80e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:06.951997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1422cd-ef24-4919-833a-ef70a75c80e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:06.954259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1422cd-ef24-4919-833a-ef70a75c80e8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:06.999032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1140/2248] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:32:06 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:32:07.003197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74540995-4379-4bf3-87f7-420f81ee8629 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:07.031546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74540995-4379-4bf3-87f7-420f81ee8629 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:07.118895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47511b-7596-4d3a-8889-7ce1b1f242ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:07.119764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47511b-7596-4d3a-8889-7ce1b1f242ec admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:07.120803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47511b-7596-4d3a-8889-7ce1b1f242ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:07.121545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47511b-7596-4d3a-8889-7ce1b1f242ec admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:07.122458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47511b-7596-4d3a-8889-7ce1b1f242ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:07.124446 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47511b-7596-4d3a-8889-7ce1b1f242ec admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:07.156810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1109/2249] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:32:07 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:32:07.158328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1422cd-ef24-4919-833a-ef70a75c80e8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:07.173135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1422cd-ef24-4919-833a-ef70a75c80e8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:07.178520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecd543-5561-441e-b603-a9c43da7fac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:07.179198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecd543-5561-441e-b603-a9c43da7fac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:07.179484 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecd543-5561-441e-b603-a9c43da7fac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:07.179758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecd543-5561-441e-b603-a9c43da7fac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:07.180497 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecd543-5561-441e-b603-a9c43da7fac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:07.182138 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecd543-5561-441e-b603-a9c43da7fac3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:07.200916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1141/2250] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:32:07.202506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47511b-7596-4d3a-8889-7ce1b1f242ec admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:07.219263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47511b-7596-4d3a-8889-7ce1b1f242ec admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:07.224732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdc836-98ff-42e7-a1c7-57ff5e8bee7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:07.225513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdc836-98ff-42e7-a1c7-57ff5e8bee7e admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:07.225885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdc836-98ff-42e7-a1c7-57ff5e8bee7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:07.226219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdc836-98ff-42e7-a1c7-57ff5e8bee7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:07.227090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdc836-98ff-42e7-a1c7-57ff5e8bee7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:07.229609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdc836-98ff-42e7-a1c7-57ff5e8bee7e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:07.241500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1110/2251] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:32:07.247546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecd543-5561-441e-b603-a9c43da7fac3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:07.262049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecd543-5561-441e-b603-a9c43da7fac3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:07.268609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238353-002d-4386-b86b-1303485065b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:07.270621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238353-002d-4386-b86b-1303485065b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:07.271365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238353-002d-4386-b86b-1303485065b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:07.272197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238353-002d-4386-b86b-1303485065b4 admin admin] PATH_INFO: `/v3/projects/7147f362728e4fc3bf3395049519e0b5/users/565810eac46f4c04b785adecf2594d7e/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:07.297475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42238353-002d-4386-b86b-1303485065b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7147f362728e4fc3bf3395049519e0b5, user_id=565810eac46f4c04b785adecf2594d7e, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:07.301348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42238353-002d-4386-b86b-1303485065b4 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:07.322242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1142/2252] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:32:07 2022] PUT /identity/v3/projects/7147f362728e4fc3bf3395049519e0b5/users/565810eac46f4c04b785adecf2594d7e/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:07.337745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d03794-e5a3-4fe3-ae1c-f80b139924ea None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:07.338218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d03794-e5a3-4fe3-ae1c-f80b139924ea None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:07.338689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d03794-e5a3-4fe3-ae1c-f80b139924ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:07.368740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:07.391243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-60d03794-e5a3-4fe3-ae1c-f80b139924ea None None] MFA Rules not processed for user `565810eac46f4c04b785adecf2594d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:07.393218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-60d03794-e5a3-4fe3-ae1c-f80b139924ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:07.485286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1111/2253] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:07.497390 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238353-002d-4386-b86b-1303485065b4 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:07.499298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-42238353-002d-4386-b86b-1303485065b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:07.597541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238353-002d-4386-b86b-1303485065b4 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:07.602164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc50c76-a428-4ce8-af1b-cb9852f74102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:07.602831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc50c76-a428-4ce8-af1b-cb9852f74102 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:07.603122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc50c76-a428-4ce8-af1b-cb9852f74102 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:07.603391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc50c76-a428-4ce8-af1b-cb9852f74102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:07.628320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc50c76-a428-4ce8-af1b-cb9852f74102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:07.630549 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc50c76-a428-4ce8-af1b-cb9852f74102 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:07.645975 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1143/2254] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:17.481456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=28 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:17.639295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:19.107652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42877f5f-04a9-427d-9c95-5842095559d4 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:19.108112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42877f5f-04a9-427d-9c95-5842095559d4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:19.108966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42877f5f-04a9-427d-9c95-5842095559d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:19.148852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-42877f5f-04a9-427d-9c95-5842095559d4 None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:19.153068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-42877f5f-04a9-427d-9c95-5842095559d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:19.267487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1112/2255] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:32:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:19.297586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc50c76-a428-4ce8-af1b-cb9852f74102 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:19.300969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8fc50c76-a428-4ce8-af1b-cb9852f74102 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:19.443769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc50c76-a428-4ce8-af1b-cb9852f74102 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:19.454012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f30ca-6eef-4d47-b68b-97ac67fb6160 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:19.454748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f30ca-6eef-4d47-b68b-97ac67fb6160 service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:19.455036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f30ca-6eef-4d47-b68b-97ac67fb6160 service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:19.455338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f30ca-6eef-4d47-b68b-97ac67fb6160 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:19.465168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f30ca-6eef-4d47-b68b-97ac67fb6160 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:19.466776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f30ca-6eef-4d47-b68b-97ac67fb6160 service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:19.475908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1144/2256] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:32:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:32:29.259175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=28 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:29.472779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=32 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:46.555413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42877f5f-04a9-427d-9c95-5842095559d4 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:46.556945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-42877f5f-04a9-427d-9c95-5842095559d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:46.665425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42877f5f-04a9-427d-9c95-5842095559d4 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:46.670144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:46.671011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:46.671288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:46.671559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] PATH_INFO: `/v3/users/c5fe3a6de74c4402b1e2d7f9cab4852f` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:46.672893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5fe3a6de74c4402b1e2d7f9cab4852f)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:46.674839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:46.738113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5fe3a6de74c4402b1e2d7f9cab4852f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:46.739141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5fe3a6de74c4402b1e2d7f9cab4852f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:46.753793 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5fe3a6de74c4402b1e2d7f9cab4852f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:46.754630 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5fe3a6de74c4402b1e2d7f9cab4852f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:46.771672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5fe3a6de74c4402b1e2d7f9cab4852f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:46.789907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1113/2257] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:32:46 2022] DELETE /identity/v3/users/c5fe3a6de74c4402b1e2d7f9cab4852f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:46.795360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccf16b-7fb8-433d-87d3-f386c7818b84 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:46.796398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccf16b-7fb8-433d-87d3-f386c7818b84 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:46.796825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccf16b-7fb8-433d-87d3-f386c7818b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:46.825688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-91ccf16b-7fb8-433d-87d3-f386c7818b84 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:46.826894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-91ccf16b-7fb8-433d-87d3-f386c7818b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:46.915862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1145/2258] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:46.935676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:46.937237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:47.047543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389eec4-8423-4c2e-84c6-07314c10a094 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:47.053404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc5b57-8f9e-4b9f-b7a2-ce45292a3f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:47.054541 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc5b57-8f9e-4b9f-b7a2-ce45292a3f1d service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:47.055077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc5b57-8f9e-4b9f-b7a2-ce45292a3f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:47.055459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc5b57-8f9e-4b9f-b7a2-ce45292a3f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:47.078559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc5b57-8f9e-4b9f-b7a2-ce45292a3f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:47.080102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc5b57-8f9e-4b9f-b7a2-ce45292a3f1d service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:47.094163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1114/2259] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:47.961519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccf16b-7fb8-433d-87d3-f386c7818b84 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:47.966918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-91ccf16b-7fb8-433d-87d3-f386c7818b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:47.983898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccf16b-7fb8-433d-87d3-f386c7818b84 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:47.989033 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:47.989982 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:47.990495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:47.990977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] PATH_INFO: `/v3/projects/2d5867b29ac444959b10c0653ed1ef15` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:47.992476 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d5867b29ac444959b10c0653ed1ef15)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:47.994605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:48.054321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d5867b29ac444959b10c0653ed1ef15', 'request_id': 'req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.064467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d5867b29ac444959b10c0653ed1ef15', 'request_id': 'req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.070413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d5867b29ac444959b10c0653ed1ef15', 'request_id': 'req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.085868 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] The token cache is being invalidate because project 2d5867b29ac444959b10c0653ed1ef15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:32:48.086163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.086845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-11d7c039-fbd4-42f5-ba05-04dc247c6ffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.088546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1146/2260] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:32:47 2022] DELETE /identity/v3/projects/2d5867b29ac444959b10c0653ed1ef15 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:48.093642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f53f0-1973-44d9-b6c8-1b0d7e532334 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.094133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f53f0-1973-44d9-b6c8-1b0d7e532334 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.094873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f53f0-1973-44d9-b6c8-1b0d7e532334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.116183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-657f53f0-1973-44d9-b6c8-1b0d7e532334 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:48.117684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-657f53f0-1973-44d9-b6c8-1b0d7e532334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:48.151613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:48.201016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1115/2261] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:48.210156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724c18-756f-4a1c-b70e-49e52a748c64 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.210594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724c18-756f-4a1c-b70e-49e52a748c64 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.210714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724c18-756f-4a1c-b70e-49e52a748c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.234566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-32724c18-756f-4a1c-b70e-49e52a748c64 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:48.236009 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-32724c18-756f-4a1c-b70e-49e52a748c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:48.250020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1147/2262] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:48.253834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f53f0-1973-44d9-b6c8-1b0d7e532334 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:48.266009 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f53f0-1973-44d9-b6c8-1b0d7e532334 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:48.281247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639dbf-1fc0-44f0-88a6-b44df5bb82e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:48.282043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639dbf-1fc0-44f0-88a6-b44df5bb82e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.282416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639dbf-1fc0-44f0-88a6-b44df5bb82e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.282757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639dbf-1fc0-44f0-88a6-b44df5bb82e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.283854 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98639dbf-1fc0-44f0-88a6-b44df5bb82e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:32:48.284326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98639dbf-1fc0-44f0-88a6-b44df5bb82e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:48.286415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98639dbf-1fc0-44f0-88a6-b44df5bb82e2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:48.300978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1116/2263] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:32:48.304038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724c18-756f-4a1c-b70e-49e52a748c64 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:48.322418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724c18-756f-4a1c-b70e-49e52a748c64 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:48.328428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bec16-827c-4d60-94a1-fe7a871f5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:48.329231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bec16-827c-4d60-94a1-fe7a871f5f14 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.329581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bec16-827c-4d60-94a1-fe7a871f5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.329928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bec16-827c-4d60-94a1-fe7a871f5f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.330723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bec16-827c-4d60-94a1-fe7a871f5f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:48.332764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bec16-827c-4d60-94a1-fe7a871f5f14 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:48.359624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1148/2264] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:32:48 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:32:48.363761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639dbf-1fc0-44f0-88a6-b44df5bb82e2 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:48.378427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639dbf-1fc0-44f0-88a6-b44df5bb82e2 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:48.444969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e1a87-1744-473a-b7dc-e01a88729466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:48.445667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e1a87-1744-473a-b7dc-e01a88729466 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.445959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e1a87-1744-473a-b7dc-e01a88729466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.446239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e1a87-1744-473a-b7dc-e01a88729466 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.446968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e1a87-1744-473a-b7dc-e01a88729466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:48.449673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e1a87-1744-473a-b7dc-e01a88729466 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:48.471688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1117/2265] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:32:48 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:32:48.475888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bec16-827c-4d60-94a1-fe7a871f5f14 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:48.486599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bec16-827c-4d60-94a1-fe7a871f5f14 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:48.491261 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9961a-3dcf-41c2-bdb8-8e7ba8e24359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:48.491920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9961a-3dcf-41c2-bdb8-8e7ba8e24359 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.492202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9961a-3dcf-41c2-bdb8-8e7ba8e24359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.492484 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9961a-3dcf-41c2-bdb8-8e7ba8e24359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.493247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9961a-3dcf-41c2-bdb8-8e7ba8e24359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:48.494880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9961a-3dcf-41c2-bdb8-8e7ba8e24359 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:48.505360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1149/2266] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:32:48.509333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e1a87-1744-473a-b7dc-e01a88729466 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:48.520056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e1a87-1744-473a-b7dc-e01a88729466 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:48.524780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee64a13-d6ce-493a-980b-b9d4200b4029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:48.525467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee64a13-d6ce-493a-980b-b9d4200b4029 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.525751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee64a13-d6ce-493a-980b-b9d4200b4029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.526700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee64a13-d6ce-493a-980b-b9d4200b4029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.527520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee64a13-d6ce-493a-980b-b9d4200b4029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:48.529165 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee64a13-d6ce-493a-980b-b9d4200b4029 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:48.539279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1118/2267] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:32:48.543974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9961a-3dcf-41c2-bdb8-8e7ba8e24359 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:48.553971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9961a-3dcf-41c2-bdb8-8e7ba8e24359 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:48.559474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83328236-4d01-421f-b129-a9a5d0c03db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:48.560361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83328236-4d01-421f-b129-a9a5d0c03db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.560647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83328236-4d01-421f-b129-a9a5d0c03db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.561473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83328236-4d01-421f-b129-a9a5d0c03db7 admin admin] PATH_INFO: `/v3/projects/daeedbb18d174046848923a2fb745418/users/95b5140fd89749729f05293c380fa0c0/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.575681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83328236-4d01-421f-b129-a9a5d0c03db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daeedbb18d174046848923a2fb745418, user_id=95b5140fd89749729f05293c380fa0c0, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:48.578384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83328236-4d01-421f-b129-a9a5d0c03db7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:48.581182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee64a13-d6ce-493a-980b-b9d4200b4029 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:48.583372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-2ee64a13-d6ce-493a-980b-b9d4200b4029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:48.597181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee64a13-d6ce-493a-980b-b9d4200b4029 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:48.598313 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1150/2268] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:32:48 2022] PUT /identity/v3/projects/daeedbb18d174046848923a2fb745418/users/95b5140fd89749729f05293c380fa0c0/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:48.609299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e55605-b8c2-46e2-b0c7-a286135322b1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.609591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e55605-b8c2-46e2-b0c7-a286135322b1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.609871 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e55605-b8c2-46e2-b0c7-a286135322b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.642938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-90e55605-b8c2-46e2-b0c7-a286135322b1 None None] MFA Rules not processed for user `95b5140fd89749729f05293c380fa0c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:48.644161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-90e55605-b8c2-46e2-b0c7-a286135322b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:48.654439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:48.680480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:48.681564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.681926 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.682266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] PATH_INFO: `/v3/users/398779837495418fb9953b928209b88f` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.684432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=398779837495418fb9953b928209b88f)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:48.686804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:48.735086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1151/2269] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:32:48.736391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:48.747167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e55605-b8c2-46e2-b0c7-a286135322b1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:48.748457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-90e55605-b8c2-46e2-b0c7-a286135322b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:48.762995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398779837495418fb9953b928209b88f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.764260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398779837495418fb9953b928209b88f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.774263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398779837495418fb9953b928209b88f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.775069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398779837495418fb9953b928209b88f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.795888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-30a04599-0ded-4313-83ba-8655e01c44c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398779837495418fb9953b928209b88f'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:48.820979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1119/2270] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:32:48 2022] DELETE /identity/v3/users/398779837495418fb9953b928209b88f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:32:48.828999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a7e05-5c45-484c-a349-4e9ea0f81136 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.829608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a7e05-5c45-484c-a349-4e9ea0f81136 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.830570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a7e05-5c45-484c-a349-4e9ea0f81136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.861146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-290a7e05-5c45-484c-a349-4e9ea0f81136 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:48.862500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-290a7e05-5c45-484c-a349-4e9ea0f81136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:48.865606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e55605-b8c2-46e2-b0c7-a286135322b1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:48.947616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:48.950641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.951049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.952121 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.954268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:48.965787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1120/2271] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:48.975516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a7e05-5c45-484c-a349-4e9ea0f81136 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:48.976920 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-290a7e05-5c45-484c-a349-4e9ea0f81136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:49.020734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:49.023027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:49.029206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:49.034862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1152/2272] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:49.037326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a7e05-5c45-484c-a349-4e9ea0f81136 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:49.041012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8ab9-3a37-48c4-b624-9b75a8e3ed7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:49.041709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8ab9-3a37-48c4-b624-9b75a8e3ed7d service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:49.041998 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8ab9-3a37-48c4-b624-9b75a8e3ed7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:49.042382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8ab9-3a37-48c4-b624-9b75a8e3ed7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:49.052644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d8ab9-3a37-48c4-b624-9b75a8e3ed7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:49.058839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d8ab9-3a37-48c4-b624-9b75a8e3ed7d service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:49.059852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:49.061318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:49.065207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1121/2273] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:32:49.076216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8479f75-bc15-4cdc-b4da-0e7f5594994e service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:49.082813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:49.082813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:49.082813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:49.082813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] PATH_INFO: `/v3/users/95b5140fd89749729f05293c380fa0c0` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:49.083468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95b5140fd89749729f05293c380fa0c0)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:49.085457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:49.140618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95b5140fd89749729f05293c380fa0c0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.154089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95b5140fd89749729f05293c380fa0c0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.162289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95b5140fd89749729f05293c380fa0c0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.162934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95b5140fd89749729f05293c380fa0c0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.163552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95b5140fd89749729f05293c380fa0c0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.176306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1153/2274] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:32:49 2022] DELETE /identity/v3/users/95b5140fd89749729f05293c380fa0c0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:32:49.181464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f9332-9f47-49b8-ac9b-2993b999f58b None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:49.181961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f9332-9f47-49b8-ac9b-2993b999f58b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:49.182509 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f9332-9f47-49b8-ac9b-2993b999f58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:49.200695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-fb7f9332-9f47-49b8-ac9b-2993b999f58b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:49.201924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-fb7f9332-9f47-49b8-ac9b-2993b999f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:49.291299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:49.308751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1122/2275] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:49.318761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:49.320778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:49.409642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f9332-9f47-49b8-ac9b-2993b999f58b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:49.422684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-fb7f9332-9f47-49b8-ac9b-2993b999f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:49.443244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f9332-9f47-49b8-ac9b-2993b999f58b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:49.455961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:49.456844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:49.457160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:49.457433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] PATH_INFO: `/v3/projects/d91b049126b7462ca8b5d0ed79a1c6ef` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:49.458778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d91b049126b7462ca8b5d0ed79a1c6ef)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:49.460496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ec317-5bdb-4587-9b4b-8156d71f9839 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:49.460720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:49.465328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db68be9-5c03-4315-9b87-581a0618d5ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:49.466685 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db68be9-5c03-4315-9b87-581a0618d5ef service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:49.467070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db68be9-5c03-4315-9b87-581a0618d5ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:49.467372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db68be9-5c03-4315-9b87-581a0618d5ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:49.496200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db68be9-5c03-4315-9b87-581a0618d5ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:49.498987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db68be9-5c03-4315-9b87-581a0618d5ef service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:49.514356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1154/2276] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:49.541514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd91b049126b7462ca8b5d0ed79a1c6ef', 'request_id': 'req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.560241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd91b049126b7462ca8b5d0ed79a1c6ef', 'request_id': 'req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.570532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd91b049126b7462ca8b5d0ed79a1c6ef', 'request_id': 'req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.589833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] The token cache is being invalidate because project d91b049126b7462ca8b5d0ed79a1c6ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:32:49.589833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.590701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6622cfe0-baf1-4cae-8248-e6648a7a7bc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:49.592621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1123/2277] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:32:49 2022] DELETE /identity/v3/projects/d91b049126b7462ca8b5d0ed79a1c6ef => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:32:50.030473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db68be9-5c03-4315-9b87-581a0618d5ef service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:50.031905 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7db68be9-5c03-4315-9b87-581a0618d5ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:50.046391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db68be9-5c03-4315-9b87-581a0618d5ef service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:50.138204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:50.138204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:50.138204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:50.138204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] PATH_INFO: `/v3/projects/daeedbb18d174046848923a2fb745418` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:50.138995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daeedbb18d174046848923a2fb745418)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:50.138995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:50.166156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:50.202776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daeedbb18d174046848923a2fb745418', 'request_id': 'req-15a67463-9bab-40f5-831e-51b60c99a530'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:50.237985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daeedbb18d174046848923a2fb745418', 'request_id': 'req-15a67463-9bab-40f5-831e-51b60c99a530'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:50.249123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daeedbb18d174046848923a2fb745418', 'request_id': 'req-15a67463-9bab-40f5-831e-51b60c99a530'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:50.273047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] The token cache is being invalidate because project daeedbb18d174046848923a2fb745418 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:32:50.273047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:50.277358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-15a67463-9bab-40f5-831e-51b60c99a530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:32:50.292371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1155/2278] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:32:50 2022] DELETE /identity/v3/projects/daeedbb18d174046848923a2fb745418 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:32:51.362396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8488b7b-e856-4f43-b4e9-9036cbaaf342 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:51.362860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8488b7b-e856-4f43-b4e9-9036cbaaf342 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:51.362995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8488b7b-e856-4f43-b4e9-9036cbaaf342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:51.391087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e8488b7b-e856-4f43-b4e9-9036cbaaf342 None None] MFA Rules not processed for user `5074b9fa045543b9a62bf751e736d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:51.393049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e8488b7b-e856-4f43-b4e9-9036cbaaf342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:51.473909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1124/2279] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:32:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:52.572260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da2f7-d985-4cb3-8079-2a21266694db None None] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:52.572721 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da2f7-d985-4cb3-8079-2a21266694db None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:52.572830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da2f7-d985-4cb3-8079-2a21266694db None None] PATH_INFO: `/` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:52.575184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1156/2280] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:32:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:52.582352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbd3b0-5e34-41e6-8167-ea29bf72642f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:52.582800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbd3b0-5e34-41e6-8167-ea29bf72642f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:52.582900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbd3b0-5e34-41e6-8167-ea29bf72642f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:52.642669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4cfbd3b0-5e34-41e6-8167-ea29bf72642f None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:52.644001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4cfbd3b0-5e34-41e6-8167-ea29bf72642f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:52.707386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:52.737756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1125/2281] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:32:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:52.749814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48da2f7-d985-4cb3-8079-2a21266694db None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:52.751522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d48da2f7-d985-4cb3-8079-2a21266694db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:52.831876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48da2f7-d985-4cb3-8079-2a21266694db None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:52.835376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f9ce1-925d-49c8-a858-e783a2e68449 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb0e53d4b3774959820ac72824b3d4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:52.836029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f9ce1-925d-49c8-a858-e783a2e68449 service glance] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:52.836314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f9ce1-925d-49c8-a858-e783a2e68449 service glance] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:52.836597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f9ce1-925d-49c8-a858-e783a2e68449 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:52.848040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f9ce1-925d-49c8-a858-e783a2e68449 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:52.849528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f9ce1-925d-49c8-a858-e783a2e68449 service glance] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:52.861387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1157/2282] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:52.941224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f4fd-d9a1-4f00-bfbf-92d4380811a6 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:52.941511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f4fd-d9a1-4f00-bfbf-92d4380811a6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:52.941908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f4fd-d9a1-4f00-bfbf-92d4380811a6 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:52.943341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1126/2283] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:52.948621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94d01d-d733-4035-9643-12ac46b47bb9 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:52.948969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94d01d-d733-4035-9643-12ac46b47bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:52.949246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94d01d-d733-4035-9643-12ac46b47bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:52.966583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1c94d01d-d733-4035-9643-12ac46b47bb9 None None] MFA Rules not processed for user `68d85e20dc884cd0bad5bb6c304776ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:52.967758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1c94d01d-d733-4035-9643-12ac46b47bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:53.024446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1158/2284] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:32:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:53.036459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f4fd-d9a1-4f00-bfbf-92d4380811a6 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:53.038282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a568f4fd-d9a1-4f00-bfbf-92d4380811a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:53.060835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f4fd-d9a1-4f00-bfbf-92d4380811a6 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:53.064553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa25545-870e-4ec8-ab4f-1543298749c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:53.065246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa25545-870e-4ec8-ab4f-1543298749c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:53.065558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa25545-870e-4ec8-ab4f-1543298749c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:53.065850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa25545-870e-4ec8-ab4f-1543298749c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:53.074669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa25545-870e-4ec8-ab4f-1543298749c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:53.076120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa25545-870e-4ec8-ab4f-1543298749c9 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:53.084597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1127/2285] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:53.126962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94d01d-d733-4035-9643-12ac46b47bb9 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:53.128398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1c94d01d-d733-4035-9643-12ac46b47bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:53.169826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed1166-c17c-4140-97af-75adf25d6038 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:53.170189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed1166-c17c-4140-97af-75adf25d6038 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:53.170477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed1166-c17c-4140-97af-75adf25d6038 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:53.172187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1128/2286] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:53.176298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:53.184141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d7fb9-62ca-43c5-be25-3cce0dfc9dff None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:53.186163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d7fb9-62ca-43c5-be25-3cce0dfc9dff None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:53.186458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d7fb9-62ca-43c5-be25-3cce0dfc9dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:53.208968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-099d7fb9-62ca-43c5-be25-3cce0dfc9dff None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:53.210168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-099d7fb9-62ca-43c5-be25-3cce0dfc9dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:53.220573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1129/2287] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:32:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:53.233337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d7fb9-62ca-43c5-be25-3cce0dfc9dff None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:53.234863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-099d7fb9-62ca-43c5-be25-3cce0dfc9dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:53.236496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94d01d-d733-4035-9643-12ac46b47bb9 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:53.241878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed7410-719a-4ca2-bdc5-75364506400d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:53.242860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed7410-719a-4ca2-bdc5-75364506400d service placement] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:53.242860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed7410-719a-4ca2-bdc5-75364506400d service placement] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:53.243091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed7410-719a-4ca2-bdc5-75364506400d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:53.245591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8fed7410-719a-4ca2-bdc5-75364506400d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:53.247762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d7fb9-62ca-43c5-be25-3cce0dfc9dff None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:53.251406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9693c-949b-4384-9e2b-145e7986f6be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:53.252051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9693c-949b-4384-9e2b-145e7986f6be service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:53.252332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9693c-949b-4384-9e2b-145e7986f6be service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:53.252609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9693c-949b-4384-9e2b-145e7986f6be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:53.259013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed7410-719a-4ca2-bdc5-75364506400d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:53.260759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed7410-719a-4ca2-bdc5-75364506400d service placement] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:53.266414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9693c-949b-4384-9e2b-145e7986f6be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:53.267870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9693c-949b-4384-9e2b-145e7986f6be service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:53.272098 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1159/2288] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:53.280522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1130/2289] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:32:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:33:02.488060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed7410-719a-4ca2-bdc5-75364506400d service placement] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:02.521521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed7410-719a-4ca2-bdc5-75364506400d service placement] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:02.527924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:02.528595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:02.528891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:02.529285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:02.531244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:02.610404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:02.611795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:02.622891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1160/2290] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:33:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:03.274658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=25 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:06.566791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279daac-e156-408f-8f21-0c355ae181e4 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:06.567397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279daac-e156-408f-8f21-0c355ae181e4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:06.567557 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279daac-e156-408f-8f21-0c355ae181e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:06.630190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-7279daac-e156-408f-8f21-0c355ae181e4 None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:06.631565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7279daac-e156-408f-8f21-0c355ae181e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:06.715115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1131/2291] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:33:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:06.741273 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:06.742848 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:06.855055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9053a-767f-42e0-ab32-4afc96a97909 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:06.863669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457711-da66-4ed2-a538-dcb8577f732a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:06.864540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457711-da66-4ed2-a538-dcb8577f732a service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:06.864954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457711-da66-4ed2-a538-dcb8577f732a service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:06.865336 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457711-da66-4ed2-a538-dcb8577f732a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:06.875204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55457711-da66-4ed2-a538-dcb8577f732a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:06.877696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55457711-da66-4ed2-a538-dcb8577f732a service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:06.897652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1161/2292] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:33:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:33:10.829125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279daac-e156-408f-8f21-0c355ae181e4 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:10.830707 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7279daac-e156-408f-8f21-0c355ae181e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:10.900555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279daac-e156-408f-8f21-0c355ae181e4 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:10.905218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:10.906079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:10.906357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:10.906646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] PATH_INFO: `/v3/users/565810eac46f4c04b785adecf2594d7e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:10.907805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=565810eac46f4c04b785adecf2594d7e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:10.909757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:10.968481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '565810eac46f4c04b785adecf2594d7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:10.969143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '565810eac46f4c04b785adecf2594d7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:10.976433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '565810eac46f4c04b785adecf2594d7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:10.977156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '565810eac46f4c04b785adecf2594d7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:10.989688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '565810eac46f4c04b785adecf2594d7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:10.991540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:11.011614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1132/2293] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:33:10 2022] DELETE /identity/v3/users/565810eac46f4c04b785adecf2594d7e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:33:11.013521 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa47ba-8d9e-419c-94ae-83860d0fb2e2 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:11.014016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa47ba-8d9e-419c-94ae-83860d0fb2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:11.014170 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa47ba-8d9e-419c-94ae-83860d0fb2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:11.032453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b8aa47ba-8d9e-419c-94ae-83860d0fb2e2 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:11.033689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b8aa47ba-8d9e-419c-94ae-83860d0fb2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:11.038668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:11.103654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1162/2294] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:33:11.115422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:11.116855 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:11.199314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0910f-fd7e-4a94-988f-b4c52393fc93 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:11.203256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2741-8fc8-49c5-80e4-67c70a075987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:11.203913 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2741-8fc8-49c5-80e4-67c70a075987 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:11.204198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2741-8fc8-49c5-80e4-67c70a075987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:11.204482 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2741-8fc8-49c5-80e4-67c70a075987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:11.215546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da2741-8fc8-49c5-80e4-67c70a075987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:11.217274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da2741-8fc8-49c5-80e4-67c70a075987 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:11.238799 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1133/2295] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:33:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:11.635368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa47ba-8d9e-419c-94ae-83860d0fb2e2 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:11.637620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b8aa47ba-8d9e-419c-94ae-83860d0fb2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:11.652138 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa47ba-8d9e-419c-94ae-83860d0fb2e2 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:11.666707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:11.667584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:11.667882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:11.668137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] PATH_INFO: `/v3/projects/7147f362728e4fc3bf3395049519e0b5` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:11.669384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7147f362728e4fc3bf3395049519e0b5)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:11.671303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:11.725558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7147f362728e4fc3bf3395049519e0b5', 'request_id': 'req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:11.732350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7147f362728e4fc3bf3395049519e0b5', 'request_id': 'req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:11.739460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7147f362728e4fc3bf3395049519e0b5', 'request_id': 'req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:11.757069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] The token cache is being invalidate because project 7147f362728e4fc3bf3395049519e0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:33:11.757561 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:11.758797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:33:11.761242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1163/2296] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:33:11 2022] DELETE /identity/v3/projects/7147f362728e4fc3bf3395049519e0b5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:33:21.223984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=28 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:21.753389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:31.614815 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15887-9eb4-4efc-9391-5f56091284af None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:31.615203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15887-9eb4-4efc-9391-5f56091284af None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:31.616052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15887-9eb4-4efc-9391-5f56091284af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:31.633542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-1bd15887-9eb4-4efc-9391-5f56091284af None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:31.634781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1bd15887-9eb4-4efc-9391-5f56091284af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:31.694081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1134/2297] 10.222.0.30 () {58 vars in 967 bytes} [Mon May 23 15:33:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:31.707069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:31.708456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:31.777663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738a5d3-3ed0-4f6c-b548-efed03aa1fcb admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:31.782607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c064d-9a7f-439e-9278-aa74b1b97f71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:31.783251 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c064d-9a7f-439e-9278-aa74b1b97f71 service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:31.783535 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c064d-9a7f-439e-9278-aa74b1b97f71 service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:31.783813 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c064d-9a7f-439e-9278-aa74b1b97f71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:31.791952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c064d-9a7f-439e-9278-aa74b1b97f71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:31.793584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c064d-9a7f-439e-9278-aa74b1b97f71 service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:31.801587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1164/2298] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:33:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:33:41.687291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=25 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:41.798408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=32 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:03.766870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60452237-5b41-45eb-9db0-d210309ddd69 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:03.767405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60452237-5b41-45eb-9db0-d210309ddd69 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:03.767503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60452237-5b41-45eb-9db0-d210309ddd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:03.784915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-60452237-5b41-45eb-9db0-d210309ddd69 None None] MFA Rules not processed for user `06431315dc8040cbbafbe83a3ed8e0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:03.786105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-60452237-5b41-45eb-9db0-d210309ddd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:03.840938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1135/2299] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:34:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:34:13.834443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=25 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:24.218438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285a3af-9403-4b06-8f71-a11c55ad4496 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:24.219031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285a3af-9403-4b06-8f71-a11c55ad4496 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:24.219031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285a3af-9403-4b06-8f71-a11c55ad4496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:24.238598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-3285a3af-9403-4b06-8f71-a11c55ad4496 None None] MFA Rules not processed for user `139e3e42acff4c4ca8df0a7c4fc81053`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:24.239801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3285a3af-9403-4b06-8f71-a11c55ad4496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:24.299320 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1165/2300] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:34.291656 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=25 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:37.077565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370e824-8a49-40fe-b353-824a5b6fa668 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:37.078085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370e824-8a49-40fe-b353-824a5b6fa668 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:37.078182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370e824-8a49-40fe-b353-824a5b6fa668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:37.099103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b370e824-8a49-40fe-b353-824a5b6fa668 None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:37.100328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b370e824-8a49-40fe-b353-824a5b6fa668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:37.110286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1136/2301] 10.222.0.30 () {58 vars in 967 bytes} [Mon May 23 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:34:37.159464 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285a3af-9403-4b06-8f71-a11c55ad4496 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:37.172377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285a3af-9403-4b06-8f71-a11c55ad4496 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:37.177380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa06a53-d9b3-4eea-a5ea-a83c21a08017 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:37.178027 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa06a53-d9b3-4eea-a5ea-a83c21a08017 service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:37.178307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa06a53-d9b3-4eea-a5ea-a83c21a08017 service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:37.178580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa06a53-d9b3-4eea-a5ea-a83c21a08017 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:37.187209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa06a53-d9b3-4eea-a5ea-a83c21a08017 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:37.188535 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa06a53-d9b3-4eea-a5ea-a83c21a08017 service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:37.196765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1166/2302] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:34:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:34:41.192542 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947a43d-12a5-47ac-96dd-ab0c31a6514e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:41.193674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947a43d-12a5-47ac-96dd-ab0c31a6514e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:41.194189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947a43d-12a5-47ac-96dd-ab0c31a6514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:41.217997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-6947a43d-12a5-47ac-96dd-ab0c31a6514e None None] MFA Rules not processed for user `5074b9fa045543b9a62bf751e736d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:41.219536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6947a43d-12a5-47ac-96dd-ab0c31a6514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:41.295216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1137/2303] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:45.479268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa06a53-d9b3-4eea-a5ea-a83c21a08017 service swift] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:45.481644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-aaa06a53-d9b3-4eea-a5ea-a83c21a08017 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:45.557863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa06a53-d9b3-4eea-a5ea-a83c21a08017 service swift] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:45.563775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:45.564698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:45.565008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:45.565517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] PATH_INFO: `/v3/users/ceed468da0c844b688f5f91768b0dd96` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:45.566789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceed468da0c844b688f5f91768b0dd96)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:45.569218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:45.630781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceed468da0c844b688f5f91768b0dd96'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:45.647054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceed468da0c844b688f5f91768b0dd96'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:45.657100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceed468da0c844b688f5f91768b0dd96'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:45.657741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceed468da0c844b688f5f91768b0dd96'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:45.658332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceed468da0c844b688f5f91768b0dd96'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:45.671979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1167/2304] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:34:45 2022] DELETE /identity/v3/users/ceed468da0c844b688f5f91768b0dd96 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:34:45.851983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947a43d-12a5-47ac-96dd-ab0c31a6514e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:45.863751 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947a43d-12a5-47ac-96dd-ab0c31a6514e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:45.934527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:45.935460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:45.935745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:45.936017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] PATH_INFO: `/v3/projects/591802cf5c044932940307866f388790` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:45.937340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591802cf5c044932940307866f388790)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:45.939348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:45.993742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591802cf5c044932940307866f388790', 'request_id': 'req-24422fb1-f1bf-41d2-a435-a6a99ea53888'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.001177 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591802cf5c044932940307866f388790', 'request_id': 'req-24422fb1-f1bf-41d2-a435-a6a99ea53888'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.009862 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591802cf5c044932940307866f388790', 'request_id': 'req-24422fb1-f1bf-41d2-a435-a6a99ea53888'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.022142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] The token cache is being invalidate because project 591802cf5c044932940307866f388790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:34:46.022430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.023060 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.024907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1138/2305] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:34:45 2022] DELETE /identity/v3/projects/591802cf5c044932940307866f388790 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:34:46.028798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:46.030645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:46.043639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8e9e6-91fd-4a47-b045-e554b56007ce admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:46.106296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:46.107227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:46.107618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:46.107911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] PATH_INFO: `/v3/users/df1d0eecdec24af7ab1bab9c2b825a85` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:46.109156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df1d0eecdec24af7ab1bab9c2b825a85)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:46.111312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:46.117590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:46.161173 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df1d0eecdec24af7ab1bab9c2b825a85'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.173662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df1d0eecdec24af7ab1bab9c2b825a85'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.181162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df1d0eecdec24af7ab1bab9c2b825a85'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.181783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df1d0eecdec24af7ab1bab9c2b825a85'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.182402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df1d0eecdec24af7ab1bab9c2b825a85'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.195647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1168/2306] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:34:46 2022] DELETE /identity/v3/users/df1d0eecdec24af7ab1bab9c2b825a85 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:34:46.525430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:46.526864 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:46.539640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24422fb1-f1bf-41d2-a435-a6a99ea53888 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:46.572415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:46.605919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:46.607057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:46.607700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:46.607885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] PATH_INFO: `/v3/projects/06253b6d930e47fc97c4a4c0376e0d5e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:46.609318 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06253b6d930e47fc97c4a4c0376e0d5e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:46.612985 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:46.659379 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06253b6d930e47fc97c4a4c0376e0d5e', 'request_id': 'req-6718a089-abab-41c4-92cd-995d8ea54074'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.668408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06253b6d930e47fc97c4a4c0376e0d5e', 'request_id': 'req-6718a089-abab-41c4-92cd-995d8ea54074'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.678440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06253b6d930e47fc97c4a4c0376e0d5e', 'request_id': 'req-6718a089-abab-41c4-92cd-995d8ea54074'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.697208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] The token cache is being invalidate because project 06253b6d930e47fc97c4a4c0376e0d5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:34:46.697622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.698852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:34:46.700712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1139/2307] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:34:46 2022] DELETE /identity/v3/projects/06253b6d930e47fc97c4a4c0376e0d5e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:34:47.231907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:47.233553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:47.291346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df41e31-0430-4e59-bf94-1ba49aa70691 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:47.342456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febcd4c-f328-4777-9abb-30037e23bcf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:47.343139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febcd4c-f328-4777-9abb-30037e23bcf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:47.343451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febcd4c-f328-4777-9abb-30037e23bcf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:47.343729 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febcd4c-f328-4777-9abb-30037e23bcf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:47.347311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0febcd4c-f328-4777-9abb-30037e23bcf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:47.400141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:47.422777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febcd4c-f328-4777-9abb-30037e23bcf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:47.424748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febcd4c-f328-4777-9abb-30037e23bcf1 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:47.436267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1169/2308] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:34:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:48.621042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:48.622454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:48.635489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718a089-abab-41c4-92cd-995d8ea54074 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:48.639081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ca7e3-49c2-4d2f-b6fc-812dfdce5961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:48.639713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ca7e3-49c2-4d2f-b6fc-812dfdce5961 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:48.639994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ca7e3-49c2-4d2f-b6fc-812dfdce5961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:48.640271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ca7e3-49c2-4d2f-b6fc-812dfdce5961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:48.642016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6c1ca7e3-49c2-4d2f-b6fc-812dfdce5961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:48.655300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ca7e3-49c2-4d2f-b6fc-812dfdce5961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:48.656734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ca7e3-49c2-4d2f-b6fc-812dfdce5961 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:48.667523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1140/2309] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:34:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:54.560439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb07a77-7332-4d6f-bed9-3a311970f985 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:54.560969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb07a77-7332-4d6f-bed9-3a311970f985 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:54.561193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb07a77-7332-4d6f-bed9-3a311970f985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:54.577979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-fdb07a77-7332-4d6f-bed9-3a311970f985 None None] MFA Rules not processed for user `d0ae34a029c14c6f826456414ac0670d`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:54.579159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fdb07a77-7332-4d6f-bed9-3a311970f985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:54.639315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1170/2310] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:34:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:54.938151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ca7e3-49c2-4d2f-b6fc-812dfdce5961 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:54.946688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ca7e3-49c2-4d2f-b6fc-812dfdce5961 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:54.950296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe35da8-2974-4547-8faa-d428e76cbbc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:54.950925 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe35da8-2974-4547-8faa-d428e76cbbc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:54.952072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe35da8-2974-4547-8faa-d428e76cbbc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:54.952355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe35da8-2974-4547-8faa-d428e76cbbc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:54.954155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0fe35da8-2974-4547-8faa-d428e76cbbc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:54.963573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe35da8-2974-4547-8faa-d428e76cbbc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:54.965045 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe35da8-2974-4547-8faa-d428e76cbbc8 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:54.972900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1141/2311] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:34:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:55.285789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb07a77-7332-4d6f-bed9-3a311970f985 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:55.287358 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fdb07a77-7332-4d6f-bed9-3a311970f985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:55.359794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb07a77-7332-4d6f-bed9-3a311970f985 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:55.364770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:55.365470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:55.365762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:55.366037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:55.367796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:55.379690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:55.381319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:55.391281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1171/2312] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:34:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:04.213547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe35da8-2974-4547-8faa-d428e76cbbc8 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:04.222116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe35da8-2974-4547-8faa-d428e76cbbc8 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:04.225752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:04.226381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:04.226659 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:04.226924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:04.228662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:04.237821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:04.239240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:04.247304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1142/2313] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:05.386990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=65 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:08.019957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:08.021522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:08.083793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7146c6f-534b-4e39-b753-7e68dfd7c107 service placement] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:08.088372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035178a-a5b7-442b-a5c7-3627d673536a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:08.089343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035178a-a5b7-442b-a5c7-3627d673536a service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:08.089775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035178a-a5b7-442b-a5c7-3627d673536a service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:08.090193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035178a-a5b7-442b-a5c7-3627d673536a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:08.094243 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e035178a-a5b7-442b-a5c7-3627d673536a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:08.162269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035178a-a5b7-442b-a5c7-3627d673536a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:08.163823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035178a-a5b7-442b-a5c7-3627d673536a service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:08.175370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1172/2314] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:13.174316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:13.176466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:13.195345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d3967-d211-4f22-b9ea-e07af4fc208b service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:13.201038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:13.202705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:13.202705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:13.202923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:13.204749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:13.219006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:13.220681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:13.242435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1143/2315] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:15.769628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d2d90-b8e1-4370-910e-b5a8e01e6942 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:15.770402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d2d90-b8e1-4370-910e-b5a8e01e6942 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:15.770498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d2d90-b8e1-4370-910e-b5a8e01e6942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:15.787732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ef9d2d90-b8e1-4370-910e-b5a8e01e6942 None None] MFA Rules not processed for user `06431315dc8040cbbafbe83a3ed8e0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:15.788930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ef9d2d90-b8e1-4370-910e-b5a8e01e6942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:15.795763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1173/2316] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:16.108945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:16.110141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:16.121670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc5fce-e38b-4b9c-b75c-86996f3d5eac service nova] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:16.125497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:16.126160 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:16.126441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:16.126714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:16.128407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:16.137590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:16.139023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:16.151679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1144/2317] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:35:21.891731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d2d90-b8e1-4370-910e-b5a8e01e6942 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:21.894100 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ef9d2d90-b8e1-4370-910e-b5a8e01e6942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:21.923054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d2d90-b8e1-4370-910e-b5a8e01e6942 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:21.932176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6b32b-8108-48b3-b455-caf8406c3f8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:21.933262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6b32b-8108-48b3-b455-caf8406c3f8e service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:21.933777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6b32b-8108-48b3-b455-caf8406c3f8e service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:21.934294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6b32b-8108-48b3-b455-caf8406c3f8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:21.937413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-70e6b32b-8108-48b3-b455-caf8406c3f8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:22.019753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6b32b-8108-48b3-b455-caf8406c3f8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:22.021660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6b32b-8108-48b3-b455-caf8406c3f8e service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:22.031705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1174/2318] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:25.584560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:25.586031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:25.593562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:25.600109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67922590-563e-41b6-b10e-03741f2f28b0 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:25.605229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f460f-9e6d-4451-8308-706f6429ec45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:25.605885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f460f-9e6d-4451-8308-706f6429ec45 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:25.606190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f460f-9e6d-4451-8308-706f6429ec45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:25.606964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f460f-9e6d-4451-8308-706f6429ec45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:25.608779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-663f460f-9e6d-4451-8308-706f6429ec45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:25.681103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f460f-9e6d-4451-8308-706f6429ec45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:25.682491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f460f-9e6d-4451-8308-706f6429ec45 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:25.694156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1145/2319] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:32.027124 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=79 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:35.689397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=27 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:44.116554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6b32b-8108-48b3-b455-caf8406c3f8e service nova] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:44.132721 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6b32b-8108-48b3-b455-caf8406c3f8e service nova] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:44.140159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:44.142134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:44.142598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:44.143070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:44.145435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:44.169778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:44.173447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:44.211869 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1175/2320] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:44.335781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b6f1-7571-4865-a94f-2520b79cbd5e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:44.336816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b6f1-7571-4865-a94f-2520b79cbd5e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:44.337415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b6f1-7571-4865-a94f-2520b79cbd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:44.361021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-bf44b6f1-7571-4865-a94f-2520b79cbd5e None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:44.362266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-bf44b6f1-7571-4865-a94f-2520b79cbd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:44.419385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1146/2321] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:35:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:44.465980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:44.467383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:44.540584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d11f9-b56e-4bd6-8102-18e35b940370 service nova] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:44.545653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87303a4a-b3d2-4ef8-9386-35b24eeac7ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:44.546315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87303a4a-b3d2-4ef8-9386-35b24eeac7ff service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:44.546601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87303a4a-b3d2-4ef8-9386-35b24eeac7ff service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:44.546882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87303a4a-b3d2-4ef8-9386-35b24eeac7ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:44.556732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87303a4a-b3d2-4ef8-9386-35b24eeac7ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:44.558424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87303a4a-b3d2-4ef8-9386-35b24eeac7ff service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:44.567924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1176/2322] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:35:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:35:45.437772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44b6f1-7571-4865-a94f-2520b79cbd5e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:45.449675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44b6f1-7571-4865-a94f-2520b79cbd5e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:45.454236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6ac6d-fc69-405e-ace7-31e6979cda1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:45.455021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6ac6d-fc69-405e-ace7-31e6979cda1f service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:45.455380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6ac6d-fc69-405e-ace7-31e6979cda1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:45.455729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6ac6d-fc69-405e-ace7-31e6979cda1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:45.457619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1eb6ac6d-fc69-405e-ace7-31e6979cda1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:45.530340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6ac6d-fc69-405e-ace7-31e6979cda1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:45.532329 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6ac6d-fc69-405e-ace7-31e6979cda1f service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:45.544403 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1147/2323] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:49.809517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87303a4a-b3d2-4ef8-9386-35b24eeac7ff service swift] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:49.811103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-87303a4a-b3d2-4ef8-9386-35b24eeac7ff service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:49.835644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87303a4a-b3d2-4ef8-9386-35b24eeac7ff service swift] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:49.843465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:49.845286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:49.846191 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:49.846929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] PATH_INFO: `/v3/users/5074b9fa045543b9a62bf751e736d0df` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:49.849049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5074b9fa045543b9a62bf751e736d0df)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:49.853094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:49.924122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5074b9fa045543b9a62bf751e736d0df'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:49.943868 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5074b9fa045543b9a62bf751e736d0df'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:49.951899 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5074b9fa045543b9a62bf751e736d0df'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:49.952630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5074b9fa045543b9a62bf751e736d0df'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:49.956005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-dc8d980b-87c3-4c80-ae26-fef84abfccbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5074b9fa045543b9a62bf751e736d0df'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:49.968448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1177/2324] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:35:49 2022] DELETE /identity/v3/users/5074b9fa045543b9a62bf751e736d0df => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:50.173442 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6ac6d-fc69-405e-ace7-31e6979cda1f service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:50.184472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6ac6d-fc69-405e-ace7-31e6979cda1f service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:50.272387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:50.273406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.273795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.274178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] PATH_INFO: `/v3/projects/0712b807a4584f7b8f019ceb65ff1e14` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.275607 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0712b807a4584f7b8f019ceb65ff1e14)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:50.279101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:50.332004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:50.340915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0712b807a4584f7b8f019ceb65ff1e14', 'request_id': 'req-d05946b1-db57-49a9-9855-ccaa42fa8f3e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:50.349968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0712b807a4584f7b8f019ceb65ff1e14', 'request_id': 'req-d05946b1-db57-49a9-9855-ccaa42fa8f3e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:50.360280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0712b807a4584f7b8f019ceb65ff1e14', 'request_id': 'req-d05946b1-db57-49a9-9855-ccaa42fa8f3e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:50.374473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] The token cache is being invalidate because project 0712b807a4584f7b8f019ceb65ff1e14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:35:50.374818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:50.375504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d05946b1-db57-49a9-9855-ccaa42fa8f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:50.378022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1148/2325] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:35:50 2022] DELETE /identity/v3/projects/0712b807a4584f7b8f019ceb65ff1e14 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:50.382893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716452a-b0c1-4c16-b1cc-8ca0c2c0cac5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.383184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716452a-b0c1-4c16-b1cc-8ca0c2c0cac5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.383462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716452a-b0c1-4c16-b1cc-8ca0c2c0cac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.403730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f716452a-b0c1-4c16-b1cc-8ca0c2c0cac5 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:50.404995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f716452a-b0c1-4c16-b1cc-8ca0c2c0cac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:50.461770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:50.480923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1178/2326] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:50.488903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e6abc-0c7b-4f2a-b9b0-18956f27d2c9 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.489197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e6abc-0c7b-4f2a-b9b0-18956f27d2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.489465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e6abc-0c7b-4f2a-b9b0-18956f27d2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.512326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-595e6abc-0c7b-4f2a-b9b0-18956f27d2c9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:50.513144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-595e6abc-0c7b-4f2a-b9b0-18956f27d2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:50.525353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1149/2327] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:50.532763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716452a-b0c1-4c16-b1cc-8ca0c2c0cac5 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:50.543572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716452a-b0c1-4c16-b1cc-8ca0c2c0cac5 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:50.548575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e46150-c148-428a-81d5-2b9f7f3ee802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:50.549302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e46150-c148-428a-81d5-2b9f7f3ee802 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.549586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e46150-c148-428a-81d5-2b9f7f3ee802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.549865 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e46150-c148-428a-81d5-2b9f7f3ee802 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.551130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e46150-c148-428a-81d5-2b9f7f3ee802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:35:50.551130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e46150-c148-428a-81d5-2b9f7f3ee802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:50.552591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e46150-c148-428a-81d5-2b9f7f3ee802 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:50.578393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1179/2328] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:35:50.580927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e6abc-0c7b-4f2a-b9b0-18956f27d2c9 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:50.589642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e6abc-0c7b-4f2a-b9b0-18956f27d2c9 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:50.598030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d766b5-a9fa-478d-a941-835e6f5641a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:50.598715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d766b5-a9fa-478d-a941-835e6f5641a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.598996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d766b5-a9fa-478d-a941-835e6f5641a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.599270 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d766b5-a9fa-478d-a941-835e6f5641a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.599974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d766b5-a9fa-478d-a941-835e6f5641a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:50.602006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d766b5-a9fa-478d-a941-835e6f5641a4 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:50.636222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1150/2329] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:35:50 2022] POST /identity/v3/projects => generated 403 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:50.641451 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e46150-c148-428a-81d5-2b9f7f3ee802 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:50.657993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e46150-c148-428a-81d5-2b9f7f3ee802 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:50.725119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b145e3-f99e-4c58-8bfb-a9d1ee1a3858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:50.725972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b145e3-f99e-4c58-8bfb-a9d1ee1a3858 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.726711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b145e3-f99e-4c58-8bfb-a9d1ee1a3858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.726711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b145e3-f99e-4c58-8bfb-a9d1ee1a3858 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.727444 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b145e3-f99e-4c58-8bfb-a9d1ee1a3858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:50.729317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b145e3-f99e-4c58-8bfb-a9d1ee1a3858 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:50.763199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1180/2330] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:35:50 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:50.768352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d766b5-a9fa-478d-a941-835e6f5641a4 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:50.780605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d766b5-a9fa-478d-a941-835e6f5641a4 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:50.788218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8bde7-8bd7-4f64-8740-8f573cd4e9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:50.788944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8bde7-8bd7-4f64-8740-8f573cd4e9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.789224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8bde7-8bd7-4f64-8740-8f573cd4e9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.789516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8bde7-8bd7-4f64-8740-8f573cd4e9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.790240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8bde7-8bd7-4f64-8740-8f573cd4e9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:50.791848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8bde7-8bd7-4f64-8740-8f573cd4e9d6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:50.805305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1151/2331] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:50.809235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b145e3-f99e-4c58-8bfb-a9d1ee1a3858 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:50.820693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b145e3-f99e-4c58-8bfb-a9d1ee1a3858 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:50.825734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1929a-9b73-4446-aa0d-29559724b782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:50.826506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1929a-9b73-4446-aa0d-29559724b782 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.826847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1929a-9b73-4446-aa0d-29559724b782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.827266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1929a-9b73-4446-aa0d-29559724b782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.828160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1929a-9b73-4446-aa0d-29559724b782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:50.832457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1929a-9b73-4446-aa0d-29559724b782 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:50.849649 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1181/2332] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:50.851155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8bde7-8bd7-4f64-8740-8f573cd4e9d6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:50.863749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8bde7-8bd7-4f64-8740-8f573cd4e9d6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:50.867853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62729b-7d55-474d-8fe6-0394890ae156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:50.868735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62729b-7d55-474d-8fe6-0394890ae156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.869046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62729b-7d55-474d-8fe6-0394890ae156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.869624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62729b-7d55-474d-8fe6-0394890ae156 admin admin] PATH_INFO: `/v3/projects/ed18973b741247a887c250ffd4b7aa56/users/ca3ad453356845508a6f891942e32cb1/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.886165 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62729b-7d55-474d-8fe6-0394890ae156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed18973b741247a887c250ffd4b7aa56, user_id=ca3ad453356845508a6f891942e32cb1, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:50.888857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62729b-7d55-474d-8fe6-0394890ae156 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:50.915211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1152/2333] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:35:50 2022] PUT /identity/v3/projects/ed18973b741247a887c250ffd4b7aa56/users/ca3ad453356845508a6f891942e32cb1/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:50.916968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9269-68db-417e-80a4-65c290efaf4c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:50.917271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9269-68db-417e-80a4-65c290efaf4c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:50.917551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9269-68db-417e-80a4-65c290efaf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:50.936718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d93f9269-68db-417e-80a4-65c290efaf4c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:50.938244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d93f9269-68db-417e-80a4-65c290efaf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:51.036338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1182/2334] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:51.053508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62729b-7d55-474d-8fe6-0394890ae156 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:51.055528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-bb62729b-7d55-474d-8fe6-0394890ae156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:51.146979 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62729b-7d55-474d-8fe6-0394890ae156 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:51.151111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1b93f-ee41-487f-9cd6-a3afae39ced4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:51.151870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b93f-ee41-487f-9cd6-a3afae39ced4 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:51.152246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b93f-ee41-487f-9cd6-a3afae39ced4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:51.152989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b93f-ee41-487f-9cd6-a3afae39ced4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:51.166880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1b93f-ee41-487f-9cd6-a3afae39ced4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:51.168702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1b93f-ee41-487f-9cd6-a3afae39ced4 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:51.185061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1153/2335] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:51.550164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9269-68db-417e-80a4-65c290efaf4c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:51.551892 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d93f9269-68db-417e-80a4-65c290efaf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:51.664797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:51.696120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9269-68db-417e-80a4-65c290efaf4c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:51.701706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae02ce-93e4-47d0-ba0b-baf5614f442d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:51.702360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae02ce-93e4-47d0-ba0b-baf5614f442d service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:51.702667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae02ce-93e4-47d0-ba0b-baf5614f442d service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:51.703002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae02ce-93e4-47d0-ba0b-baf5614f442d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:51.705278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7aae02ce-93e4-47d0-ba0b-baf5614f442d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:51.796963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae02ce-93e4-47d0-ba0b-baf5614f442d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:51.798756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae02ce-93e4-47d0-ba0b-baf5614f442d service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:51.809523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1183/2336] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:52.098627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e05b65-1e43-4012-9ea0-9edecbee6ca3 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:52.098907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e05b65-1e43-4012-9ea0-9edecbee6ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:52.099197 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e05b65-1e43-4012-9ea0-9edecbee6ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:52.135357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b6e05b65-1e43-4012-9ea0-9edecbee6ca3 None None] MFA Rules not processed for user `ca3ad453356845508a6f891942e32cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:52.136962 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b6e05b65-1e43-4012-9ea0-9edecbee6ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:52.266030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:52.287538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1154/2337] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:52.292523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae02ce-93e4-47d0-ba0b-baf5614f442d service nova] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:52.307209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae02ce-93e4-47d0-ba0b-baf5614f442d service nova] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:52.320302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3da262-8d8c-4d75-a16d-567d871668f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:52.321269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3da262-8d8c-4d75-a16d-567d871668f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:52.321779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3da262-8d8c-4d75-a16d-567d871668f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:52.321887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3da262-8d8c-4d75-a16d-567d871668f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:52.322647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3da262-8d8c-4d75-a16d-567d871668f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:52.324698 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3da262-8d8c-4d75-a16d-567d871668f7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:52.388274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1184/2338] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:35:52 2022] POST /identity/v3/projects => generated 405 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:52.389025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e05b65-1e43-4012-9ea0-9edecbee6ca3 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:52.407114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e05b65-1e43-4012-9ea0-9edecbee6ca3 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:52.493039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e49a67-ff2d-427a-bb83-9f8bea32a486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:52.495166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e49a67-ff2d-427a-bb83-9f8bea32a486 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:52.495456 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e49a67-ff2d-427a-bb83-9f8bea32a486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:52.495781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e49a67-ff2d-427a-bb83-9f8bea32a486 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:52.496484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e49a67-ff2d-427a-bb83-9f8bea32a486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:52.498495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e49a67-ff2d-427a-bb83-9f8bea32a486 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:52.528444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1155/2339] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:35:52 2022] POST /identity/v3/users => generated 309 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:52.528680 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3da262-8d8c-4d75-a16d-567d871668f7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:52.539465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3da262-8d8c-4d75-a16d-567d871668f7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:52.544852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9019cf-4fac-4827-b906-70ac45df5653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:52.545546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9019cf-4fac-4827-b906-70ac45df5653 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:52.545829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9019cf-4fac-4827-b906-70ac45df5653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:52.546108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9019cf-4fac-4827-b906-70ac45df5653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:52.546826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9019cf-4fac-4827-b906-70ac45df5653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:52.551528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9019cf-4fac-4827-b906-70ac45df5653 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:52.564490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1185/2340] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:52.568908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e49a67-ff2d-427a-bb83-9f8bea32a486 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:52.579444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e49a67-ff2d-427a-bb83-9f8bea32a486 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:52.585072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30601f-6f41-445b-a98a-20fa678e2700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:52.589295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30601f-6f41-445b-a98a-20fa678e2700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:52.589295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30601f-6f41-445b-a98a-20fa678e2700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:52.589295 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30601f-6f41-445b-a98a-20fa678e2700 admin admin] PATH_INFO: `/v3/projects/ab3b1bd0268e4e41bd4d31411304e358/users/67e320baab0c4375bcc35cd9dd328f90/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:52.602873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30601f-6f41-445b-a98a-20fa678e2700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab3b1bd0268e4e41bd4d31411304e358, user_id=67e320baab0c4375bcc35cd9dd328f90, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:52.605746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30601f-6f41-445b-a98a-20fa678e2700 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:52.623721 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1156/2341] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:35:52 2022] PUT /identity/v3/projects/ab3b1bd0268e4e41bd4d31411304e358/users/67e320baab0c4375bcc35cd9dd328f90/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:52.628122 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9019cf-4fac-4827-b906-70ac45df5653 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:52.640731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9019cf-4fac-4827-b906-70ac45df5653 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:52.708159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27de6f9-75a6-4d1a-9845-aad21f2736a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:52.709175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27de6f9-75a6-4d1a-9845-aad21f2736a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:52.709612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27de6f9-75a6-4d1a-9845-aad21f2736a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:52.710123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27de6f9-75a6-4d1a-9845-aad21f2736a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:52.711684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27de6f9-75a6-4d1a-9845-aad21f2736a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:52.713539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27de6f9-75a6-4d1a-9845-aad21f2736a2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:52.725422 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1186/2342] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:52.729643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30601f-6f41-445b-a98a-20fa678e2700 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:52.744397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30601f-6f41-445b-a98a-20fa678e2700 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:52.752757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ae08-4fda-462e-a243-7666976ebe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:52.753955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ae08-4fda-462e-a243-7666976ebe9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:52.754422 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ae08-4fda-462e-a243-7666976ebe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:52.754857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ae08-4fda-462e-a243-7666976ebe9c admin admin] PATH_INFO: `/v3/projects/ab3b1bd0268e4e41bd4d31411304e358/users/67e320baab0c4375bcc35cd9dd328f90/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:52.757112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1ae08-4fda-462e-a243-7666976ebe9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab3b1bd0268e4e41bd4d31411304e358, user_id=67e320baab0c4375bcc35cd9dd328f90, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:52.760117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1ae08-4fda-462e-a243-7666976ebe9c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:52.774739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1157/2343] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:35:52 2022] PUT /identity/v3/projects/ab3b1bd0268e4e41bd4d31411304e358/users/67e320baab0c4375bcc35cd9dd328f90/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:35:53.742200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3e7a6-f13b-4bb2-931f-709f68f2569f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:53.742751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3e7a6-f13b-4bb2-931f-709f68f2569f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:53.742960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3e7a6-f13b-4bb2-931f-709f68f2569f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:53.801742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-acc3e7a6-f13b-4bb2-931f-709f68f2569f None None] MFA Rules not processed for user `67e320baab0c4375bcc35cd9dd328f90`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:53.803545 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-acc3e7a6-f13b-4bb2-931f-709f68f2569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:53.872723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:53.920801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1187/2344] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:53 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:55.222702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ae08-4fda-462e-a243-7666976ebe9c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:55.286859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ae08-4fda-462e-a243-7666976ebe9c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:55.291535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:55.292399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:55.292682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:55.292995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] PATH_INFO: `/v3/users/ca3ad453356845508a6f891942e32cb1` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:55.294198 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca3ad453356845508a6f891942e32cb1)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:55.296140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:55.341505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca3ad453356845508a6f891942e32cb1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.342146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca3ad453356845508a6f891942e32cb1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.346679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:55.350699 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca3ad453356845508a6f891942e32cb1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.351747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca3ad453356845508a6f891942e32cb1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.365546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca3ad453356845508a6f891942e32cb1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.378138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1158/2345] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:35:55 2022] DELETE /identity/v3/users/ca3ad453356845508a6f891942e32cb1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:55.551301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3e7a6-f13b-4bb2-931f-709f68f2569f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:55.566065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3e7a6-f13b-4bb2-931f-709f68f2569f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:55.630127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:55.630989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:55.631291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:55.631563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] PATH_INFO: `/v3/projects/ed18973b741247a887c250ffd4b7aa56` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:55.632719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed18973b741247a887c250ffd4b7aa56)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:55.634673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:55.681617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed18973b741247a887c250ffd4b7aa56', 'request_id': 'req-3904ce77-ee9e-4235-b053-768893cbd764'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.687758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed18973b741247a887c250ffd4b7aa56', 'request_id': 'req-3904ce77-ee9e-4235-b053-768893cbd764'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.693953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed18973b741247a887c250ffd4b7aa56', 'request_id': 'req-3904ce77-ee9e-4235-b053-768893cbd764'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.707096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] The token cache is being invalidate because project ed18973b741247a887c250ffd4b7aa56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:35:55.707394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.708003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.709605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1188/2346] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:35:55 2022] DELETE /identity/v3/projects/ed18973b741247a887c250ffd4b7aa56 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:55.713592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:55.715068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:55.728416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a42bd1-968e-42f1-8346-ba2ad3420fb6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:55.786480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:55.787358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:55.787650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:55.787928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] PATH_INFO: `/v3/users/67e320baab0c4375bcc35cd9dd328f90` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:55.789156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=67e320baab0c4375bcc35cd9dd328f90)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:55.791106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:55.842016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67e320baab0c4375bcc35cd9dd328f90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.842948 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67e320baab0c4375bcc35cd9dd328f90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.852111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67e320baab0c4375bcc35cd9dd328f90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.852818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67e320baab0c4375bcc35cd9dd328f90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.865872 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-84b6e185-7ff7-4240-9cd3-3165d81d83ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67e320baab0c4375bcc35cd9dd328f90'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:55.879669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1159/2347] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:35:55 2022] DELETE /identity/v3/users/67e320baab0c4375bcc35cd9dd328f90 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:56.070076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.073333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:56.090758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904ce77-ee9e-4235-b053-768893cbd764 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:56.202929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:56.204485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.205232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.205232 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] PATH_INFO: `/v3/projects/ab3b1bd0268e4e41bd4d31411304e358` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.206421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab3b1bd0268e4e41bd4d31411304e358)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:56.208323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:56.221904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:56.268554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab3b1bd0268e4e41bd4d31411304e358', 'request_id': 'req-9c3caa82-1384-471f-baf8-c09ae7e34bfb'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:56.275944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab3b1bd0268e4e41bd4d31411304e358', 'request_id': 'req-9c3caa82-1384-471f-baf8-c09ae7e34bfb'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:56.283387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab3b1bd0268e4e41bd4d31411304e358', 'request_id': 'req-9c3caa82-1384-471f-baf8-c09ae7e34bfb'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:56.296514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] The token cache is being invalidate because project ab3b1bd0268e4e41bd4d31411304e358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:35:56.296801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:56.297501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-9c3caa82-1384-471f-baf8-c09ae7e34bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:56.299229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1189/2348] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:35:56 2022] DELETE /identity/v3/projects/ab3b1bd0268e4e41bd4d31411304e358 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:56.304027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f5710-a96d-4d37-8258-3122cf162a3d None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.304508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f5710-a96d-4d37-8258-3122cf162a3d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.304735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f5710-a96d-4d37-8258-3122cf162a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.323924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-247f5710-a96d-4d37-8258-3122cf162a3d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:56.325185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-247f5710-a96d-4d37-8258-3122cf162a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:56.370973 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:56.390286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1160/2349] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:56.398861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cd6fa-5bf8-4551-86f5-aefe840ab648 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.399177 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cd6fa-5bf8-4551-86f5-aefe840ab648 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.399454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cd6fa-5bf8-4551-86f5-aefe840ab648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.418709 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-459cd6fa-5bf8-4551-86f5-aefe840ab648 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:56.420126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-459cd6fa-5bf8-4551-86f5-aefe840ab648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:56.428369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1190/2350] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:56.432827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f5710-a96d-4d37-8258-3122cf162a3d None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.443945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f5710-a96d-4d37-8258-3122cf162a3d None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:56.448792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a268b1-8e40-4ce1-b5de-980e5d83d945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:56.449672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a268b1-8e40-4ce1-b5de-980e5d83d945 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.450491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a268b1-8e40-4ce1-b5de-980e5d83d945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.450813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a268b1-8e40-4ce1-b5de-980e5d83d945 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.452142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a268b1-8e40-4ce1-b5de-980e5d83d945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:35:56.452529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a268b1-8e40-4ce1-b5de-980e5d83d945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:56.455735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a268b1-8e40-4ce1-b5de-980e5d83d945 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:56.472561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1161/2351] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:35:56.473714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cd6fa-5bf8-4551-86f5-aefe840ab648 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.483519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cd6fa-5bf8-4551-86f5-aefe840ab648 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:56.488860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b36aa4-50b2-4149-8597-ca8f7835649f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:56.489489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b36aa4-50b2-4149-8597-ca8f7835649f admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.489796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b36aa4-50b2-4149-8597-ca8f7835649f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.490074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b36aa4-50b2-4149-8597-ca8f7835649f admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.490829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b36aa4-50b2-4149-8597-ca8f7835649f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:56.492797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b36aa4-50b2-4149-8597-ca8f7835649f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:56.520462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1191/2352] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:35:56 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:56.524578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a268b1-8e40-4ce1-b5de-980e5d83d945 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.534481 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a268b1-8e40-4ce1-b5de-980e5d83d945 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:56.589174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7a3c-f869-42fc-886b-602188d5b3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:56.589841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7a3c-f869-42fc-886b-602188d5b3ce admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.590119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7a3c-f869-42fc-886b-602188d5b3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.590386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7a3c-f869-42fc-886b-602188d5b3ce admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.591080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed7a3c-f869-42fc-886b-602188d5b3ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:56.593113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed7a3c-f869-42fc-886b-602188d5b3ce admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:56.614653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1162/2353] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:35:56 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:56.618853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b36aa4-50b2-4149-8597-ca8f7835649f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.629822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b36aa4-50b2-4149-8597-ca8f7835649f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:56.634374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb82078-28d8-40d3-b646-46e7617fcbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:56.635310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb82078-28d8-40d3-b646-46e7617fcbda admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.635623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb82078-28d8-40d3-b646-46e7617fcbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.635897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb82078-28d8-40d3-b646-46e7617fcbda admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.636777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb82078-28d8-40d3-b646-46e7617fcbda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:56.638392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb82078-28d8-40d3-b646-46e7617fcbda admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:56.649738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1192/2354] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:56.653566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7a3c-f869-42fc-886b-602188d5b3ce admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.663453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7a3c-f869-42fc-886b-602188d5b3ce admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:56.668240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c54327-96b7-4336-b93d-6e8f17ee9c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:56.668956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c54327-96b7-4336-b93d-6e8f17ee9c6a admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.669240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c54327-96b7-4336-b93d-6e8f17ee9c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.669541 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c54327-96b7-4336-b93d-6e8f17ee9c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.670298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c54327-96b7-4336-b93d-6e8f17ee9c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:56.671902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c54327-96b7-4336-b93d-6e8f17ee9c6a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:56.682039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1163/2355] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:56.686188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb82078-28d8-40d3-b646-46e7617fcbda admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.696927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb82078-28d8-40d3-b646-46e7617fcbda admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:56.701613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998e06f-6a18-41f6-9f89-90eed98a0a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:56.702488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e06f-6a18-41f6-9f89-90eed98a0a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.702773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e06f-6a18-41f6-9f89-90eed98a0a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.703054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e06f-6a18-41f6-9f89-90eed98a0a64 admin admin] PATH_INFO: `/v3/projects/0ea39bc2a0d443ed970266ba5a4ca69a/users/d12ee1899ee84d9f93e93cbba99602b4/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.718310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998e06f-6a18-41f6-9f89-90eed98a0a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea39bc2a0d443ed970266ba5a4ca69a, user_id=d12ee1899ee84d9f93e93cbba99602b4, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:56.721256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998e06f-6a18-41f6-9f89-90eed98a0a64 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:56.734466 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1193/2356] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:35:56 2022] PUT /identity/v3/projects/0ea39bc2a0d443ed970266ba5a4ca69a/users/d12ee1899ee84d9f93e93cbba99602b4/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:56.744687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f187324-e93c-4c43-a1a8-e2f85ce30c53 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.745020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f187324-e93c-4c43-a1a8-e2f85ce30c53 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.745299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f187324-e93c-4c43-a1a8-e2f85ce30c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.772968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-1f187324-e93c-4c43-a1a8-e2f85ce30c53 None None] MFA Rules not processed for user `d12ee1899ee84d9f93e93cbba99602b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:56.774017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1f187324-e93c-4c43-a1a8-e2f85ce30c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:56.837283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1164/2357] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:56.841744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998e06f-6a18-41f6-9f89-90eed98a0a64 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.906017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998e06f-6a18-41f6-9f89-90eed98a0a64 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:56.910854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab112e-22a8-4551-966b-cc9111abce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:56.911592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab112e-22a8-4551-966b-cc9111abce0c admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.911910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab112e-22a8-4551-966b-cc9111abce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.912215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab112e-22a8-4551-966b-cc9111abce0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.913054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab112e-22a8-4551-966b-cc9111abce0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:56.915150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab112e-22a8-4551-966b-cc9111abce0c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:56.950944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1194/2358] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:35:56 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:56.955122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f187324-e93c-4c43-a1a8-e2f85ce30c53 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.965620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f187324-e93c-4c43-a1a8-e2f85ce30c53 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:57.006969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:57.023242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9baf2-d2ec-4503-8846-8d8f006cc825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:57.023909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9baf2-d2ec-4503-8846-8d8f006cc825 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:57.024195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9baf2-d2ec-4503-8846-8d8f006cc825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:57.024479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9baf2-d2ec-4503-8846-8d8f006cc825 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:57.025204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9baf2-d2ec-4503-8846-8d8f006cc825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:57.027152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9baf2-d2ec-4503-8846-8d8f006cc825 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:57.046585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1165/2359] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:35:56 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:57.051447 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab112e-22a8-4551-966b-cc9111abce0c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:57.061802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab112e-22a8-4551-966b-cc9111abce0c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:57.067160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065f603-a0d5-4c04-ad39-b27a814d6d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:57.067823 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065f603-a0d5-4c04-ad39-b27a814d6d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:57.068108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065f603-a0d5-4c04-ad39-b27a814d6d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:57.068382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065f603-a0d5-4c04-ad39-b27a814d6d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:57.069203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065f603-a0d5-4c04-ad39-b27a814d6d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:57.071065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065f603-a0d5-4c04-ad39-b27a814d6d36 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:57.081582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1195/2360] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:57.085627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9baf2-d2ec-4503-8846-8d8f006cc825 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:57.095413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9baf2-d2ec-4503-8846-8d8f006cc825 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:57.100294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b10b1-3079-48fa-8df8-81f8043b458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:57.101312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b10b1-3079-48fa-8df8-81f8043b458f admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:57.101590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b10b1-3079-48fa-8df8-81f8043b458f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:57.101865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b10b1-3079-48fa-8df8-81f8043b458f admin admin] PATH_INFO: `/v3/projects/b818f12de2984080afb8ab101c476e9b/users/5a2a0be8f425457bb9191de92d6af4a0/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:57.118558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b10b1-3079-48fa-8df8-81f8043b458f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b818f12de2984080afb8ab101c476e9b, user_id=5a2a0be8f425457bb9191de92d6af4a0, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:57.121411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b10b1-3079-48fa-8df8-81f8043b458f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:57.134994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1166/2361] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:35:57 2022] PUT /identity/v3/projects/b818f12de2984080afb8ab101c476e9b/users/5a2a0be8f425457bb9191de92d6af4a0/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:57.138858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065f603-a0d5-4c04-ad39-b27a814d6d36 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:57.144091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:57.150486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065f603-a0d5-4c04-ad39-b27a814d6d36 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:57.209415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b4885-dee4-428c-a346-e53e6153a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:57.210126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b4885-dee4-428c-a346-e53e6153a85b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:57.210404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b4885-dee4-428c-a346-e53e6153a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:57.210721 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b4885-dee4-428c-a346-e53e6153a85b admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:57.211442 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b4885-dee4-428c-a346-e53e6153a85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:57.213229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b4885-dee4-428c-a346-e53e6153a85b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:57.228951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1196/2362] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:57.230084 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b10b1-3079-48fa-8df8-81f8043b458f admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:57.241723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b10b1-3079-48fa-8df8-81f8043b458f admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:57.246420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c511cf4-9c29-4719-9161-09836b3ec458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:57.247330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c511cf4-9c29-4719-9161-09836b3ec458 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:57.247614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c511cf4-9c29-4719-9161-09836b3ec458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:57.247911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c511cf4-9c29-4719-9161-09836b3ec458 admin admin] PATH_INFO: `/v3/projects/b818f12de2984080afb8ab101c476e9b/users/5a2a0be8f425457bb9191de92d6af4a0/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:57.249917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c511cf4-9c29-4719-9161-09836b3ec458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b818f12de2984080afb8ab101c476e9b, user_id=5a2a0be8f425457bb9191de92d6af4a0, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:57.252637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c511cf4-9c29-4719-9161-09836b3ec458 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:57.262995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1167/2363] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:35:57 2022] PUT /identity/v3/projects/b818f12de2984080afb8ab101c476e9b/users/5a2a0be8f425457bb9191de92d6af4a0/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:57.274439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dc4a5-54e8-4710-be56-e5f9bba8db2e None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:57.274730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dc4a5-54e8-4710-be56-e5f9bba8db2e None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:57.275029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dc4a5-54e8-4710-be56-e5f9bba8db2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:57.304113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d50dc4a5-54e8-4710-be56-e5f9bba8db2e None None] MFA Rules not processed for user `5a2a0be8f425457bb9191de92d6af4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:57.305386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d50dc4a5-54e8-4710-be56-e5f9bba8db2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:57.374744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1197/2364] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:57.388654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c511cf4-9c29-4719-9161-09836b3ec458 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:57.390463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7c511cf4-9c29-4719-9161-09836b3ec458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:57.449446 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c511cf4-9c29-4719-9161-09836b3ec458 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:57.462843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade3383-b266-4177-8723-978eef56b66a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:57.464729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade3383-b266-4177-8723-978eef56b66a service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:57.465037 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade3383-b266-4177-8723-978eef56b66a service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:57.465536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade3383-b266-4177-8723-978eef56b66a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:57.487173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade3383-b266-4177-8723-978eef56b66a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:57.488624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade3383-b266-4177-8723-978eef56b66a service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:57.498075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1168/2365] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:57.882153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dc4a5-54e8-4710-be56-e5f9bba8db2e None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:57.944121 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dc4a5-54e8-4710-be56-e5f9bba8db2e None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:57.948913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:57.949778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:57.950059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:57.950331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] PATH_INFO: `/v3/users/d12ee1899ee84d9f93e93cbba99602b4` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:57.951538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12ee1899ee84d9f93e93cbba99602b4)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:57.953482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:57.980788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:57.999570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12ee1899ee84d9f93e93cbba99602b4'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.011736 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12ee1899ee84d9f93e93cbba99602b4'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.019602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12ee1899ee84d9f93e93cbba99602b4'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.020260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12ee1899ee84d9f93e93cbba99602b4'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.020862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12ee1899ee84d9f93e93cbba99602b4'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.033361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1198/2366] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:35:57 2022] DELETE /identity/v3/users/d12ee1899ee84d9f93e93cbba99602b4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:58.037930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4027b-1731-44f7-9176-a9195de11d35 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:58.038225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4027b-1731-44f7-9176-a9195de11d35 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:58.038543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4027b-1731-44f7-9176-a9195de11d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:58.055966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-def4027b-1731-44f7-9176-a9195de11d35 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:58.057184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-def4027b-1731-44f7-9176-a9195de11d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:58.115567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1169/2367] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:58.125302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:58.127400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:58.183692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525f220-d278-4633-8330-33ade8a064d1 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:58.187352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fba16-e962-4cd2-ab77-abe52f891d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:58.188518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fba16-e962-4cd2-ab77-abe52f891d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:58.189042 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fba16-e962-4cd2-ab77-abe52f891d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:58.189352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fba16-e962-4cd2-ab77-abe52f891d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:58.198267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fba16-e962-4cd2-ab77-abe52f891d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:58.199692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fba16-e962-4cd2-ab77-abe52f891d9d service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:58.209706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1199/2368] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:58.507005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4027b-1731-44f7-9176-a9195de11d35 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:58.508639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-def4027b-1731-44f7-9176-a9195de11d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:58.522625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4027b-1731-44f7-9176-a9195de11d35 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:58.527443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:58.528313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:58.528595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:58.528897 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] PATH_INFO: `/v3/projects/0ea39bc2a0d443ed970266ba5a4ca69a` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:58.530097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea39bc2a0d443ed970266ba5a4ca69a)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:58.532215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:58.583597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:58.596434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea39bc2a0d443ed970266ba5a4ca69a', 'request_id': 'req-154104ae-ae58-4812-971e-6e27ee16b7b2'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.602833 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea39bc2a0d443ed970266ba5a4ca69a', 'request_id': 'req-154104ae-ae58-4812-971e-6e27ee16b7b2'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.613218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea39bc2a0d443ed970266ba5a4ca69a', 'request_id': 'req-154104ae-ae58-4812-971e-6e27ee16b7b2'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.625251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] The token cache is being invalidate because project 0ea39bc2a0d443ed970266ba5a4ca69a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:35:58.626243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.627189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.628730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1170/2369] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:35:58 2022] DELETE /identity/v3/projects/0ea39bc2a0d443ed970266ba5a4ca69a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:58.632997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fba16-e962-4cd2-ab77-abe52f891d9d service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:58.643158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fba16-e962-4cd2-ab77-abe52f891d9d service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:58.705301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:58.706289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:58.706636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:58.706974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] PATH_INFO: `/v3/users/5a2a0be8f425457bb9191de92d6af4a0` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:58.708230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2a0be8f425457bb9191de92d6af4a0)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:58.710168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:58.768594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2a0be8f425457bb9191de92d6af4a0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.770158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:58.788374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2a0be8f425457bb9191de92d6af4a0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.797430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2a0be8f425457bb9191de92d6af4a0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.799018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2a0be8f425457bb9191de92d6af4a0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.799891 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d8e0a0d1-d0b0-453b-8198-1cd582fc742a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2a0be8f425457bb9191de92d6af4a0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:58.814139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1200/2370] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:35:58 2022] DELETE /identity/v3/users/5a2a0be8f425457bb9191de92d6af4a0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:59.097029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.098472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:59.113469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154104ae-ae58-4812-971e-6e27ee16b7b2 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.177551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.178412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.178701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.178954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] PATH_INFO: `/v3/projects/b818f12de2984080afb8ab101c476e9b` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.180127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b818f12de2984080afb8ab101c476e9b)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.182071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:59.234737 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b818f12de2984080afb8ab101c476e9b', 'request_id': 'req-aefeb177-7985-4157-8d6f-6a2b341dc565'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:59.240934 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b818f12de2984080afb8ab101c476e9b', 'request_id': 'req-aefeb177-7985-4157-8d6f-6a2b341dc565'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:59.249179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b818f12de2984080afb8ab101c476e9b', 'request_id': 'req-aefeb177-7985-4157-8d6f-6a2b341dc565'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:59.262856 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] The token cache is being invalidate because project b818f12de2984080afb8ab101c476e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:35:59.263083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:59.263741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-aefeb177-7985-4157-8d6f-6a2b341dc565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:35:59.265369 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1171/2371] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:35:59 2022] DELETE /identity/v3/projects/b818f12de2984080afb8ab101c476e9b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:35:59.270974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0683aa-6919-44d6-9597-90c452f92cca None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.271511 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0683aa-6919-44d6-9597-90c452f92cca None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.271630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0683aa-6919-44d6-9597-90c452f92cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.292318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-8d0683aa-6919-44d6-9597-90c452f92cca None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:59.294394 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8d0683aa-6919-44d6-9597-90c452f92cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:59.380615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1201/2372] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:35:59.388749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3fa20-fde5-4665-a2d3-4d56eb19028f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.389071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3fa20-fde5-4665-a2d3-4d56eb19028f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.389333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3fa20-fde5-4665-a2d3-4d56eb19028f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.408578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b9d3fa20-fde5-4665-a2d3-4d56eb19028f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:59.409814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b9d3fa20-fde5-4665-a2d3-4d56eb19028f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:59.417550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1172/2373] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:59.421732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0683aa-6919-44d6-9597-90c452f92cca None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.431402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0683aa-6919-44d6-9597-90c452f92cca None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.437601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a664a-0f8e-4228-aa2f-e99a198c1b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.438998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a664a-0f8e-4228-aa2f-e99a198c1b52 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.439312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a664a-0f8e-4228-aa2f-e99a198c1b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.439616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a664a-0f8e-4228-aa2f-e99a198c1b52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.440473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a664a-0f8e-4228-aa2f-e99a198c1b52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:35:59.440805 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a664a-0f8e-4228-aa2f-e99a198c1b52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.442529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a664a-0f8e-4228-aa2f-e99a198c1b52 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:59.456817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1202/2374] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:35:59.461654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3fa20-fde5-4665-a2d3-4d56eb19028f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.472484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3fa20-fde5-4665-a2d3-4d56eb19028f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.477477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb48e-7b00-4be3-8bf0-cdb25c7031c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.478133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb48e-7b00-4be3-8bf0-cdb25c7031c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.478424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb48e-7b00-4be3-8bf0-cdb25c7031c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.478701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb48e-7b00-4be3-8bf0-cdb25c7031c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.479397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bb48e-7b00-4be3-8bf0-cdb25c7031c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.481428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bb48e-7b00-4be3-8bf0-cdb25c7031c8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:59.512297 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1173/2375] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:35:59 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:59.513801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a664a-0f8e-4228-aa2f-e99a198c1b52 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.527555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a664a-0f8e-4228-aa2f-e99a198c1b52 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.600590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6a8ff-4dae-41d9-92a6-584d5e55e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.601310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6a8ff-4dae-41d9-92a6-584d5e55e602 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.601598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6a8ff-4dae-41d9-92a6-584d5e55e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.601879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6a8ff-4dae-41d9-92a6-584d5e55e602 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.602575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6a8ff-4dae-41d9-92a6-584d5e55e602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.608013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6a8ff-4dae-41d9-92a6-584d5e55e602 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:59.628633 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1203/2376] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:35:59 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:35:59.632826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb48e-7b00-4be3-8bf0-cdb25c7031c8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.644546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb48e-7b00-4be3-8bf0-cdb25c7031c8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.649851 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63af0c3-fcb7-4834-a923-f3c05f9a546d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.650510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63af0c3-fcb7-4834-a923-f3c05f9a546d admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.650788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63af0c3-fcb7-4834-a923-f3c05f9a546d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.651070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63af0c3-fcb7-4834-a923-f3c05f9a546d admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.651781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63af0c3-fcb7-4834-a923-f3c05f9a546d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.653410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63af0c3-fcb7-4834-a923-f3c05f9a546d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:59.664672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1174/2377] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:59.669015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6a8ff-4dae-41d9-92a6-584d5e55e602 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.683473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6a8ff-4dae-41d9-92a6-584d5e55e602 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.690287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f4cc8-396e-4fe0-87e8-a3f627d200e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.690983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f4cc8-396e-4fe0-87e8-a3f627d200e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.691297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f4cc8-396e-4fe0-87e8-a3f627d200e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.691581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f4cc8-396e-4fe0-87e8-a3f627d200e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.692318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f4cc8-396e-4fe0-87e8-a3f627d200e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.694267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f4cc8-396e-4fe0-87e8-a3f627d200e7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:59.711275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1204/2378] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:59.712331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17f8ed-553c-4cd6-a136-1d07dfd7f63a None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.712617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17f8ed-553c-4cd6-a136-1d07dfd7f63a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.712910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17f8ed-553c-4cd6-a136-1d07dfd7f63a None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.714741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1175/2379] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:59.715337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f4cc8-396e-4fe0-87e8-a3f627d200e7 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.720984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb754add-adf7-4860-91fc-f3c3a065983b None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.721271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb754add-adf7-4860-91fc-f3c3a065983b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.721554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb754add-adf7-4860-91fc-f3c3a065983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.727900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f4cc8-396e-4fe0-87e8-a3f627d200e7 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.734028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36c5be-289e-43e9-956c-bfcafa9a5465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.735307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c5be-289e-43e9-956c-bfcafa9a5465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.735607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c5be-289e-43e9-956c-bfcafa9a5465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.735897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c5be-289e-43e9-956c-bfcafa9a5465 admin admin] PATH_INFO: `/v3/projects/25efd91a985d41efb27a059ab27e52d7/users/41397e11de1a4909a7f487e96665ed39/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.740435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-eb754add-adf7-4860-91fc-f3c3a065983b None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:59.741688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-eb754add-adf7-4860-91fc-f3c3a065983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:59.751567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:59.758436 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36c5be-289e-43e9-956c-bfcafa9a5465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25efd91a985d41efb27a059ab27e52d7, user_id=41397e11de1a4909a7f487e96665ed39, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.761738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36c5be-289e-43e9-956c-bfcafa9a5465 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:59.779654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1205/2380] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:35:59 2022] PUT /identity/v3/projects/25efd91a985d41efb27a059ab27e52d7/users/41397e11de1a4909a7f487e96665ed39/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:35:59.791684 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936fcdb-ee49-47ac-8f58-5712cda292ff None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.792146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936fcdb-ee49-47ac-8f58-5712cda292ff None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.792818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936fcdb-ee49-47ac-8f58-5712cda292ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.806652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:59.823144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1176/2381] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:59.824201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-9936fcdb-ee49-47ac-8f58-5712cda292ff None None] MFA Rules not processed for user `41397e11de1a4909a7f487e96665ed39`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:59.825462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9936fcdb-ee49-47ac-8f58-5712cda292ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:59.842043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb754add-adf7-4860-91fc-f3c3a065983b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.844135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-eb754add-adf7-4860-91fc-f3c3a065983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:59.903561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb754add-adf7-4860-91fc-f3c3a065983b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.908639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1206/2382] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:59.913086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936fcdb-ee49-47ac-8f58-5712cda292ff None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.956494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3eda08-dbb5-435a-9262-3d560eb740a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb0e53d4b3774959820ac72824b3d4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.957184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3eda08-dbb5-435a-9262-3d560eb740a9 service glance] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.957609 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3eda08-dbb5-435a-9262-3d560eb740a9 service glance] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.957888 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3eda08-dbb5-435a-9262-3d560eb740a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.979294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936fcdb-ee49-47ac-8f58-5712cda292ff None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.983950 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c26798-efff-46b4-a72b-c15a1d0d697c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.984609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c26798-efff-46b4-a72b-c15a1d0d697c admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.985478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c26798-efff-46b4-a72b-c15a1d0d697c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.985770 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c26798-efff-46b4-a72b-c15a1d0d697c admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.986476 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c26798-efff-46b4-a72b-c15a1d0d697c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.988396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c26798-efff-46b4-a72b-c15a1d0d697c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.017725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1207/2383] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:35:59 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:00.018740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c26798-efff-46b4-a72b-c15a1d0d697c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:00.027146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3eda08-dbb5-435a-9262-3d560eb740a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.028544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3eda08-dbb5-435a-9262-3d560eb740a9 service glance] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.039614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1177/2384] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:35:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:00.087895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c26798-efff-46b4-a72b-c15a1d0d697c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:00.093650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b5532-1614-4471-b369-387c28be4b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:00.094687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b5532-1614-4471-b369-387c28be4b9b admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.094995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b5532-1614-4471-b369-387c28be4b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.095277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b5532-1614-4471-b369-387c28be4b9b admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.095963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b5532-1614-4471-b369-387c28be4b9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.098476 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b5532-1614-4471-b369-387c28be4b9b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.112387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5a1be-3e17-45ad-82a5-0c4e36444384 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.112622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5a1be-3e17-45ad-82a5-0c4e36444384 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.113001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5a1be-3e17-45ad-82a5-0c4e36444384 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.114338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1178/2385] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:00.120924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee606d-ce9b-46bc-8738-716c2b0730e6 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.121334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee606d-ce9b-46bc-8738-716c2b0730e6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.124310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee606d-ce9b-46bc-8738-716c2b0730e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.124441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1208/2386] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:00 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:00.125370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b5532-1614-4471-b369-387c28be4b9b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:00.129311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:00.138199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b5532-1614-4471-b369-387c28be4b9b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:00.143855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b8865-53e5-4c41-bdf4-2d1418448c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:00.144268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-72ee606d-ce9b-46bc-8738-716c2b0730e6 None None] MFA Rules not processed for user `68d85e20dc884cd0bad5bb6c304776ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:00.144912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b8865-53e5-4c41-bdf4-2d1418448c04 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.144912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b8865-53e5-4c41-bdf4-2d1418448c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.145714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-72ee606d-ce9b-46bc-8738-716c2b0730e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:00.145840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b8865-53e5-4c41-bdf4-2d1418448c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.146546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b8865-53e5-4c41-bdf4-2d1418448c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.148610 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b8865-53e5-4c41-bdf4-2d1418448c04 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.156143 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:00.163652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1209/2387] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:00.167570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b8865-53e5-4c41-bdf4-2d1418448c04 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:00.182953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b8865-53e5-4c41-bdf4-2d1418448c04 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:00.190171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f7a58-62e5-402e-ae34-cb7930c38610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:00.191066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f7a58-62e5-402e-ae34-cb7930c38610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.191343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f7a58-62e5-402e-ae34-cb7930c38610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.191617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f7a58-62e5-402e-ae34-cb7930c38610 admin admin] PATH_INFO: `/v3/projects/34288882d6e648b797323efa1b1aa814/users/0bba2197e7d24a51b2dfbd22cfd6d064/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.212758 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1179/2388] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:36:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:36:00.228948 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f7a58-62e5-402e-ae34-cb7930c38610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34288882d6e648b797323efa1b1aa814, user_id=0bba2197e7d24a51b2dfbd22cfd6d064, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.231963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee606d-ce9b-46bc-8738-716c2b0730e6 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:00.232157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f7a58-62e5-402e-ae34-cb7930c38610 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.233516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-72ee606d-ce9b-46bc-8738-716c2b0730e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:00.249247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1210/2389] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:00 2022] PUT /identity/v3/projects/34288882d6e648b797323efa1b1aa814/users/0bba2197e7d24a51b2dfbd22cfd6d064/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:00.253221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f7a58-62e5-402e-ae34-cb7930c38610 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:00.259650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee606d-ce9b-46bc-8738-716c2b0730e6 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:00.318872 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e26bf-2f9e-41ff-809a-a61760af2860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:00.319530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e26bf-2f9e-41ff-809a-a61760af2860 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.319811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e26bf-2f9e-41ff-809a-a61760af2860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.320091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e26bf-2f9e-41ff-809a-a61760af2860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.327688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e26bf-2f9e-41ff-809a-a61760af2860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.329210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e26bf-2f9e-41ff-809a-a61760af2860 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.330206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f7a58-62e5-402e-ae34-cb7930c38610 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:00.334984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733abaf3-be41-4635-9419-e0473c175d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:00.335640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733abaf3-be41-4635-9419-e0473c175d69 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.335941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733abaf3-be41-4635-9419-e0473c175d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.336204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733abaf3-be41-4635-9419-e0473c175d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.336969 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733abaf3-be41-4635-9419-e0473c175d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.337680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1180/2390] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:00.338653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733abaf3-be41-4635-9419-e0473c175d69 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.350871 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1211/2391] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:36:00.355080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e26bf-2f9e-41ff-809a-a61760af2860 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:00.368715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e26bf-2f9e-41ff-809a-a61760af2860 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:00.374084 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9e980-a763-4b9d-9311-4ba03fb2cb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:00.375109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e980-a763-4b9d-9311-4ba03fb2cb4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.375484 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e980-a763-4b9d-9311-4ba03fb2cb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.376319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e980-a763-4b9d-9311-4ba03fb2cb4a admin admin] PATH_INFO: `/v3/projects/34288882d6e648b797323efa1b1aa814/users/0bba2197e7d24a51b2dfbd22cfd6d064/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.378715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9e980-a763-4b9d-9311-4ba03fb2cb4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=34288882d6e648b797323efa1b1aa814, user_id=0bba2197e7d24a51b2dfbd22cfd6d064, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.381441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9e980-a763-4b9d-9311-4ba03fb2cb4a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.393356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1181/2392] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:00 2022] PUT /identity/v3/projects/34288882d6e648b797323efa1b1aa814/users/0bba2197e7d24a51b2dfbd22cfd6d064/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:00.403895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178800-4ddc-400c-935a-1286a20a06f5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.404184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178800-4ddc-400c-935a-1286a20a06f5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.404460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178800-4ddc-400c-935a-1286a20a06f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.433024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-a9178800-4ddc-400c-935a-1286a20a06f5 None None] MFA Rules not processed for user `0bba2197e7d24a51b2dfbd22cfd6d064`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:00.434325 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a9178800-4ddc-400c-935a-1286a20a06f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:00.436383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb6bf6-1276-449a-a27a-46feacfb59e2 None None] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.436674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb6bf6-1276-449a-a27a-46feacfb59e2 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.437110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb6bf6-1276-449a-a27a-46feacfb59e2 None None] PATH_INFO: `/` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.438608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1182/2393] 10.222.0.30 () {54 vars in 921 bytes} [Mon May 23 15:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:00.444070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faa72e-923f-437b-a07e-e1ff9ae63e92 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.444503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faa72e-923f-437b-a07e-e1ff9ae63e92 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.444851 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faa72e-923f-437b-a07e-e1ff9ae63e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.464632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d6faa72e-923f-437b-a07e-e1ff9ae63e92 None None] MFA Rules not processed for user `232cfaefaf584ac59720713e02cf2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:00.465872 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d6faa72e-923f-437b-a07e-e1ff9ae63e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:00.508908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1212/2394] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:00 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:00.520235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178800-4ddc-400c-935a-1286a20a06f5 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:00.521615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a9178800-4ddc-400c-935a-1286a20a06f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:00.537170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1183/2395] 10.222.0.30 () {58 vars in 1018 bytes} [Mon May 23 15:36:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:00.548108 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faa72e-923f-437b-a07e-e1ff9ae63e92 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:00.580910 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178800-4ddc-400c-935a-1286a20a06f5 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:00.581600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:00.584794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b1a8b-8f84-4da2-8577-ef8447a4d700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:00.588927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b1a8b-8f84-4da2-8577-ef8447a4d700 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.588927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b1a8b-8f84-4da2-8577-ef8447a4d700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.588927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b1a8b-8f84-4da2-8577-ef8447a4d700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.596533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b1a8b-8f84-4da2-8577-ef8447a4d700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.597997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b1a8b-8f84-4da2-8577-ef8447a4d700 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.604559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:00.610150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faa72e-923f-437b-a07e-e1ff9ae63e92 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:00.612290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1213/2396] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:00 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:36:00.616147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3060ca-5dbd-4f87-950f-05638fc702cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:00.616838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3060ca-5dbd-4f87-950f-05638fc702cc service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.617151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3060ca-5dbd-4f87-950f-05638fc702cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.617513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3060ca-5dbd-4f87-950f-05638fc702cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.628732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3060ca-5dbd-4f87-950f-05638fc702cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.630242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3060ca-5dbd-4f87-950f-05638fc702cc service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.642270 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1184/2397] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:36:00.703103 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b1a8b-8f84-4da2-8577-ef8447a4d700 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:00.705468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b43b1a8b-8f84-4da2-8577-ef8447a4d700 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:00.717561 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b1a8b-8f84-4da2-8577-ef8447a4d700 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:00.722172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d08bc-dc61-4b1e-848b-bf92138436a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:00.723085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d08bc-dc61-4b1e-848b-bf92138436a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:00.723463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d08bc-dc61-4b1e-848b-bf92138436a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:00.723861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d08bc-dc61-4b1e-848b-bf92138436a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:00.791145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d08bc-dc61-4b1e-848b-bf92138436a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:00.792605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d08bc-dc61-4b1e-848b-bf92138436a0 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:00.802522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1214/2398] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:02.643448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfeaf1-0954-4d3b-a0cd-499a7a692f0a None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:02.643880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfeaf1-0954-4d3b-a0cd-499a7a692f0a None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:02.644005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfeaf1-0954-4d3b-a0cd-499a7a692f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:02.662321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-badfeaf1-0954-4d3b-a0cd-499a7a692f0a None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:02.663510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-badfeaf1-0954-4d3b-a0cd-499a7a692f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:02.717900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1185/2399] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:36:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:02.730767 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d08bc-dc61-4b1e-848b-bf92138436a0 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:02.732149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4b0d08bc-dc61-4b1e-848b-bf92138436a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:02.804762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d08bc-dc61-4b1e-848b-bf92138436a0 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:02.810087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3c52b-0f4a-4f8a-a150-ae210b1f81d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:02.810735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3c52b-0f4a-4f8a-a150-ae210b1f81d1 service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:02.811021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3c52b-0f4a-4f8a-a150-ae210b1f81d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:02.811300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3c52b-0f4a-4f8a-a150-ae210b1f81d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:02.819861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3c52b-0f4a-4f8a-a150-ae210b1f81d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:02.821512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3c52b-0f4a-4f8a-a150-ae210b1f81d1 service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:02.830239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1215/2400] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:36:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:36:11.380137 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfeaf1-0954-4d3b-a0cd-499a7a692f0a None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:11.461494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfeaf1-0954-4d3b-a0cd-499a7a692f0a None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:11.466070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:11.466942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:11.467218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:11.467486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] PATH_INFO: `/v3/users/41397e11de1a4909a7f487e96665ed39` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:11.468679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41397e11de1a4909a7f487e96665ed39)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:11.470859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:11.524956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41397e11de1a4909a7f487e96665ed39'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:11.525694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41397e11de1a4909a7f487e96665ed39'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:11.534466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41397e11de1a4909a7f487e96665ed39'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:11.535222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41397e11de1a4909a7f487e96665ed39'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:11.547545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41397e11de1a4909a7f487e96665ed39'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:11.557940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:11.562878 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1186/2401] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:36:11 2022] DELETE /identity/v3/users/41397e11de1a4909a7f487e96665ed39 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:11.569266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fbe15-1dab-4b93-838a-8b11fbb5dba9 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:11.569799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fbe15-1dab-4b93-838a-8b11fbb5dba9 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:11.569916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fbe15-1dab-4b93-838a-8b11fbb5dba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:11.587888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b84fbe15-1dab-4b93-838a-8b11fbb5dba9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:11.589202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b84fbe15-1dab-4b93-838a-8b11fbb5dba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:11.660674 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1216/2402] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:11.678455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:11.680185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:11.762754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6964244-0768-4b90-8edb-86e997265ac2 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:11.769543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4d35d-977b-4573-afab-8f5eb3a357fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:11.770514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4d35d-977b-4573-afab-8f5eb3a357fb service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:11.770977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4d35d-977b-4573-afab-8f5eb3a357fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:11.771435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4d35d-977b-4573-afab-8f5eb3a357fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:11.782242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4d35d-977b-4573-afab-8f5eb3a357fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:11.784085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4d35d-977b-4573-afab-8f5eb3a357fb service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:11.797468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1187/2403] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:12.120445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fbe15-1dab-4b93-838a-8b11fbb5dba9 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:12.121915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b84fbe15-1dab-4b93-838a-8b11fbb5dba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:12.135701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fbe15-1dab-4b93-838a-8b11fbb5dba9 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:12.140472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:12.141377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.141678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.142508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] PATH_INFO: `/v3/projects/25efd91a985d41efb27a059ab27e52d7` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.143814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25efd91a985d41efb27a059ab27e52d7)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:12.145818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:12.171088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:12.195248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25efd91a985d41efb27a059ab27e52d7', 'request_id': 'req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.201762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25efd91a985d41efb27a059ab27e52d7', 'request_id': 'req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.207265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25efd91a985d41efb27a059ab27e52d7', 'request_id': 'req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.219277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] The token cache is being invalidate because project 25efd91a985d41efb27a059ab27e52d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:36:12.220095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.220757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.222338 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1217/2404] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:36:12 2022] DELETE /identity/v3/projects/25efd91a985d41efb27a059ab27e52d7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:12.226591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4d35d-977b-4573-afab-8f5eb3a357fb service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:12.236820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4d35d-977b-4573-afab-8f5eb3a357fb service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:12.294667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:12.295550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.295835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.296113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] PATH_INFO: `/v3/users/0bba2197e7d24a51b2dfbd22cfd6d064` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.297344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bba2197e7d24a51b2dfbd22cfd6d064)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:12.299625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:12.347627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bba2197e7d24a51b2dfbd22cfd6d064'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.348292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bba2197e7d24a51b2dfbd22cfd6d064'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.355640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bba2197e7d24a51b2dfbd22cfd6d064'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.356939 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bba2197e7d24a51b2dfbd22cfd6d064'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.369503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f52a2665-bf11-4ce4-9b06-1fb812c8f278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bba2197e7d24a51b2dfbd22cfd6d064'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.383601 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1188/2405] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:36:12 2022] DELETE /identity/v3/users/0bba2197e7d24a51b2dfbd22cfd6d064 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:12.655238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:12.656766 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:12.671241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a6f5a-4bcb-4553-8563-cd48fa8ad9f8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:12.727936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:12.728793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.729095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.729368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] PATH_INFO: `/v3/projects/34288882d6e648b797323efa1b1aa814` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.730553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] RBAC: Authorizing `identity:delete_project(project_id=34288882d6e648b797323efa1b1aa814)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:12.732433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:12.778880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34288882d6e648b797323efa1b1aa814', 'request_id': 'req-3652e63a-4417-4cda-8c75-612f17a9495d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.784858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34288882d6e648b797323efa1b1aa814', 'request_id': 'req-3652e63a-4417-4cda-8c75-612f17a9495d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.790676 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34288882d6e648b797323efa1b1aa814', 'request_id': 'req-3652e63a-4417-4cda-8c75-612f17a9495d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.803085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] The token cache is being invalidate because project 34288882d6e648b797323efa1b1aa814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:36:12.803370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.803986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3652e63a-4417-4cda-8c75-612f17a9495d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:12.805558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1218/2406] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:36:12 2022] DELETE /identity/v3/projects/34288882d6e648b797323efa1b1aa814 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:12.811376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d643e23-7c2b-4d43-a012-4cf24db03397 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.811666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d643e23-7c2b-4d43-a012-4cf24db03397 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.811947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d643e23-7c2b-4d43-a012-4cf24db03397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.832149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-2d643e23-7c2b-4d43-a012-4cf24db03397 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:12.833373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-2d643e23-7c2b-4d43-a012-4cf24db03397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:12.850089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:12.905458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1189/2407] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:12.913457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37189e-d556-428e-919b-80034c2a8612 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.913741 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37189e-d556-428e-919b-80034c2a8612 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.914015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37189e-d556-428e-919b-80034c2a8612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.934304 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-9a37189e-d556-428e-919b-80034c2a8612 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:12.935526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9a37189e-d556-428e-919b-80034c2a8612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:12.944672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1219/2408] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:12.949239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d643e23-7c2b-4d43-a012-4cf24db03397 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:12.969204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d643e23-7c2b-4d43-a012-4cf24db03397 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:12.974884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6d1b5-6287-4b29-9d6b-d52ca06bb705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:12.975624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6d1b5-6287-4b29-9d6b-d52ca06bb705 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.975909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6d1b5-6287-4b29-9d6b-d52ca06bb705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.976185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6d1b5-6287-4b29-9d6b-d52ca06bb705 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.977000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6d1b5-6287-4b29-9d6b-d52ca06bb705 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:12.977791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6d1b5-6287-4b29-9d6b-d52ca06bb705 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:12.979340 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6d1b5-6287-4b29-9d6b-d52ca06bb705 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:12.995171 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1190/2409] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:12.999750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37189e-d556-428e-919b-80034c2a8612 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.010988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37189e-d556-428e-919b-80034c2a8612 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.017938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473f1d0-3452-4dc3-beee-76723239552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.018612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473f1d0-3452-4dc3-beee-76723239552a admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.019070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473f1d0-3452-4dc3-beee-76723239552a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.019486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473f1d0-3452-4dc3-beee-76723239552a admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.020347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473f1d0-3452-4dc3-beee-76723239552a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.022443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473f1d0-3452-4dc3-beee-76723239552a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.056481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1220/2410] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:12 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:13.060556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6d1b5-6287-4b29-9d6b-d52ca06bb705 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.092551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6d1b5-6287-4b29-9d6b-d52ca06bb705 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.192722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96a4b7-0902-4eb2-81cd-b55761f1e336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.193494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96a4b7-0902-4eb2-81cd-b55761f1e336 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.193770 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96a4b7-0902-4eb2-81cd-b55761f1e336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.194053 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96a4b7-0902-4eb2-81cd-b55761f1e336 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.194767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96a4b7-0902-4eb2-81cd-b55761f1e336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.196725 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96a4b7-0902-4eb2-81cd-b55761f1e336 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.224244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1191/2411] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:13 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:13.228486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473f1d0-3452-4dc3-beee-76723239552a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.241109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473f1d0-3452-4dc3-beee-76723239552a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.246053 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec150a2-6d97-42a4-9fde-a52fe1ff3316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.246728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec150a2-6d97-42a4-9fde-a52fe1ff3316 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.253606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec150a2-6d97-42a4-9fde-a52fe1ff3316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.253936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec150a2-6d97-42a4-9fde-a52fe1ff3316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.254705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec150a2-6d97-42a4-9fde-a52fe1ff3316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.256317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec150a2-6d97-42a4-9fde-a52fe1ff3316 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.273134 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1221/2412] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:13.277591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96a4b7-0902-4eb2-81cd-b55761f1e336 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.288536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96a4b7-0902-4eb2-81cd-b55761f1e336 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.293455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26933e-994e-4bb3-b4e9-791d22620f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.294112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26933e-994e-4bb3-b4e9-791d22620f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.294393 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26933e-994e-4bb3-b4e9-791d22620f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.294668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26933e-994e-4bb3-b4e9-791d22620f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.295388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26933e-994e-4bb3-b4e9-791d22620f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.297089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26933e-994e-4bb3-b4e9-791d22620f45 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.310236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1192/2413] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:13.314814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec150a2-6d97-42a4-9fde-a52fe1ff3316 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.320041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:13.325674 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec150a2-6d97-42a4-9fde-a52fe1ff3316 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.333675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191e10a-8990-46b5-964f-38c83c1002f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.334562 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191e10a-8990-46b5-964f-38c83c1002f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.334845 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191e10a-8990-46b5-964f-38c83c1002f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.335118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191e10a-8990-46b5-964f-38c83c1002f8 admin admin] PATH_INFO: `/v3/projects/196477e33dee45de97c12626bdcdfd55/users/3fd35be381aa4821b065933d7cfc3324/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.350285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191e10a-8990-46b5-964f-38c83c1002f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=196477e33dee45de97c12626bdcdfd55, user_id=3fd35be381aa4821b065933d7cfc3324, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.352993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191e10a-8990-46b5-964f-38c83c1002f8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.366130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1222/2414] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:13 2022] PUT /identity/v3/projects/196477e33dee45de97c12626bdcdfd55/users/3fd35be381aa4821b065933d7cfc3324/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:13.377493 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe74172-2355-4831-9f3e-041356a0f831 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.377787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe74172-2355-4831-9f3e-041356a0f831 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.378065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe74172-2355-4831-9f3e-041356a0f831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.405497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ffe74172-2355-4831-9f3e-041356a0f831 None None] MFA Rules not processed for user `3fd35be381aa4821b065933d7cfc3324`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:13.406709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ffe74172-2355-4831-9f3e-041356a0f831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:13.472035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1193/2415] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:13.476629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191e10a-8990-46b5-964f-38c83c1002f8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.536990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191e10a-8990-46b5-964f-38c83c1002f8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.541695 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb4dfb-1c80-43fb-b0ad-fe1fa5c7f6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.542298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb4dfb-1c80-43fb-b0ad-fe1fa5c7f6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.542582 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb4dfb-1c80-43fb-b0ad-fe1fa5c7f6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.542878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb4dfb-1c80-43fb-b0ad-fe1fa5c7f6d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.543546 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb4dfb-1c80-43fb-b0ad-fe1fa5c7f6d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.545498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb4dfb-1c80-43fb-b0ad-fe1fa5c7f6d5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.570556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1223/2416] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:13 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:13.574112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe74172-2355-4831-9f3e-041356a0f831 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.585322 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe74172-2355-4831-9f3e-041356a0f831 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.599182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:13.644557 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a9973-ff3b-4a20-bcc1-7bf25eb57e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.645262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a9973-ff3b-4a20-bcc1-7bf25eb57e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.645550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a9973-ff3b-4a20-bcc1-7bf25eb57e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.645830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a9973-ff3b-4a20-bcc1-7bf25eb57e60 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.646517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a9973-ff3b-4a20-bcc1-7bf25eb57e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.648719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a9973-ff3b-4a20-bcc1-7bf25eb57e60 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.669126 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1194/2417] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:13 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:13.673130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb4dfb-1c80-43fb-b0ad-fe1fa5c7f6d5 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.683965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb4dfb-1c80-43fb-b0ad-fe1fa5c7f6d5 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.689766 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8620dc2-a111-4f86-a420-38e0a1042384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.690507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8620dc2-a111-4f86-a420-38e0a1042384 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.690831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8620dc2-a111-4f86-a420-38e0a1042384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.691250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8620dc2-a111-4f86-a420-38e0a1042384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.692158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8620dc2-a111-4f86-a420-38e0a1042384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.693785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8620dc2-a111-4f86-a420-38e0a1042384 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.705973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1224/2418] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:13.709947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a9973-ff3b-4a20-bcc1-7bf25eb57e60 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.720134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a9973-ff3b-4a20-bcc1-7bf25eb57e60 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.724853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c987e2e-8f07-4e7f-a1bd-d3dbc1bf6075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.725745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c987e2e-8f07-4e7f-a1bd-d3dbc1bf6075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.726030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c987e2e-8f07-4e7f-a1bd-d3dbc1bf6075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.726305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c987e2e-8f07-4e7f-a1bd-d3dbc1bf6075 admin admin] PATH_INFO: `/v3/projects/a80388cecc59479f90cfe6ec59f7b508/users/ae83bb2bf6404f7ab9b9622d0c433583/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.741964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c987e2e-8f07-4e7f-a1bd-d3dbc1bf6075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a80388cecc59479f90cfe6ec59f7b508, user_id=ae83bb2bf6404f7ab9b9622d0c433583, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.744639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c987e2e-8f07-4e7f-a1bd-d3dbc1bf6075 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.758585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1195/2419] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:13 2022] PUT /identity/v3/projects/a80388cecc59479f90cfe6ec59f7b508/users/ae83bb2bf6404f7ab9b9622d0c433583/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:13.763534 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8620dc2-a111-4f86-a420-38e0a1042384 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.775855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8620dc2-a111-4f86-a420-38e0a1042384 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.833536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a3004-bbdd-4506-a6c2-cf2ec6b3cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.834241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3004-bbdd-4506-a6c2-cf2ec6b3cac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.834531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3004-bbdd-4506-a6c2-cf2ec6b3cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.834814 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3004-bbdd-4506-a6c2-cf2ec6b3cac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.835574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a3004-bbdd-4506-a6c2-cf2ec6b3cac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.837226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a3004-bbdd-4506-a6c2-cf2ec6b3cac3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.847415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1225/2420] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:13.851610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c987e2e-8f07-4e7f-a1bd-d3dbc1bf6075 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:13.861773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c987e2e-8f07-4e7f-a1bd-d3dbc1bf6075 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:13.866958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae674e4-d52c-470e-b1da-c41887e1fdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:13.867836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae674e4-d52c-470e-b1da-c41887e1fdab admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.868114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae674e4-d52c-470e-b1da-c41887e1fdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.868385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae674e4-d52c-470e-b1da-c41887e1fdab admin admin] PATH_INFO: `/v3/projects/a80388cecc59479f90cfe6ec59f7b508/users/ae83bb2bf6404f7ab9b9622d0c433583/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.870356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae674e4-d52c-470e-b1da-c41887e1fdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a80388cecc59479f90cfe6ec59f7b508, user_id=ae83bb2bf6404f7ab9b9622d0c433583, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:13.873044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae674e4-d52c-470e-b1da-c41887e1fdab admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:13.883942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1196/2421] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:13 2022] PUT /identity/v3/projects/a80388cecc59479f90cfe6ec59f7b508/users/ae83bb2bf6404f7ab9b9622d0c433583/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:13.894675 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b0f5c-e0e1-4139-b246-d1531eed4d16 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:13.894995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b0f5c-e0e1-4139-b246-d1531eed4d16 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:13.895276 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b0f5c-e0e1-4139-b246-d1531eed4d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:13.921585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-fa9b0f5c-e0e1-4139-b246-d1531eed4d16 None None] MFA Rules not processed for user `ae83bb2bf6404f7ab9b9622d0c433583`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:13.922878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fa9b0f5c-e0e1-4139-b246-d1531eed4d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:13.960834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:13.991153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1226/2422] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:14.002283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae674e4-d52c-470e-b1da-c41887e1fdab admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:14.003960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-eae674e4-d52c-470e-b1da-c41887e1fdab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:14.062812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae674e4-d52c-470e-b1da-c41887e1fdab admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:14.067099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a96b-8344-4f9b-ab4b-c5ecf5850356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:14.067988 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a96b-8344-4f9b-ab4b-c5ecf5850356 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:14.068123 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a96b-8344-4f9b-ab4b-c5ecf5850356 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:14.068394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a96b-8344-4f9b-ab4b-c5ecf5850356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:14.133813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392a96b-8344-4f9b-ab4b-c5ecf5850356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:14.135355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392a96b-8344-4f9b-ab4b-c5ecf5850356 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:14.146742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1197/2423] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:19.848786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b0f5c-e0e1-4139-b246-d1531eed4d16 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:19.850228 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fa9b0f5c-e0e1-4139-b246-d1531eed4d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:19.862614 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b0f5c-e0e1-4139-b246-d1531eed4d16 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:19.866150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bb0ee-f1be-4297-ade2-5d1563f4bce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:19.866791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bb0ee-f1be-4297-ade2-5d1563f4bce4 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:19.867076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bb0ee-f1be-4297-ade2-5d1563f4bce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:19.867355 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bb0ee-f1be-4297-ade2-5d1563f4bce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:19.876305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bb0ee-f1be-4297-ade2-5d1563f4bce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:19.878592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bb0ee-f1be-4297-ade2-5d1563f4bce4 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:19.891301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1227/2424] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:24.140971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:24.432114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc344be0-adc9-441b-932e-3be51e8e91f4 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:24.432404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc344be0-adc9-441b-932e-3be51e8e91f4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:24.432682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc344be0-adc9-441b-932e-3be51e8e91f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:24.450385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-bc344be0-adc9-441b-932e-3be51e8e91f4 None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:24.451623 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-bc344be0-adc9-441b-932e-3be51e8e91f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:24.511727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1198/2425] 10.222.0.30 () {58 vars in 967 bytes} [Mon May 23 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:24.525857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bb0ee-f1be-4297-ade2-5d1563f4bce4 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:24.527344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-dd7bb0ee-f1be-4297-ade2-5d1563f4bce4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:24.599881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bb0ee-f1be-4297-ade2-5d1563f4bce4 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:24.610275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3943b19-043b-42c7-8131-f26f6938b591 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:24.611528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3943b19-043b-42c7-8131-f26f6938b591 service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:24.611807 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3943b19-043b-42c7-8131-f26f6938b591 service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:24.612087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3943b19-043b-42c7-8131-f26f6938b591 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:24.621325 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3943b19-043b-42c7-8131-f26f6938b591 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:24.623002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3943b19-043b-42c7-8131-f26f6938b591 service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:24.632315 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1228/2426] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:36:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:36:34.503283 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=25 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:34.629041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=50 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:42.635130 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc344be0-adc9-441b-932e-3be51e8e91f4 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:42.730714 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc344be0-adc9-441b-932e-3be51e8e91f4 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:42.737370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:42.738247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:42.738530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:42.738805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] PATH_INFO: `/v3/users/3fd35be381aa4821b065933d7cfc3324` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:42.740162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fd35be381aa4821b065933d7cfc3324)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:42.742270 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:42.802483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fd35be381aa4821b065933d7cfc3324'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:42.803356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fd35be381aa4821b065933d7cfc3324'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:42.811837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fd35be381aa4821b065933d7cfc3324'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:42.813024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fd35be381aa4821b065933d7cfc3324'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:42.828700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fd35be381aa4821b065933d7cfc3324'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:42.851841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1199/2427] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:36:42 2022] DELETE /identity/v3/users/3fd35be381aa4821b065933d7cfc3324 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:42.856956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930457ba-51b4-4dd0-9990-26f6429040cd None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:42.857412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930457ba-51b4-4dd0-9990-26f6429040cd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:42.857572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930457ba-51b4-4dd0-9990-26f6429040cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:42.876967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-930457ba-51b4-4dd0-9990-26f6429040cd None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:42.878279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-930457ba-51b4-4dd0-9990-26f6429040cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:42.959305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1229/2428] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:36:42.974438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:42.976084 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:43.049145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc03b8e-216e-460b-b220-e5229a8aa03f admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:43.053019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4991d74-546a-4b0d-a4ab-030875dfc53b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:43.053685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4991d74-546a-4b0d-a4ab-030875dfc53b service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.054012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4991d74-546a-4b0d-a4ab-030875dfc53b service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.054221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4991d74-546a-4b0d-a4ab-030875dfc53b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.066952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4991d74-546a-4b0d-a4ab-030875dfc53b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:43.078767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4991d74-546a-4b0d-a4ab-030875dfc53b service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:43.093871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1200/2429] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:36:43.618083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930457ba-51b4-4dd0-9990-26f6429040cd None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.619981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-930457ba-51b4-4dd0-9990-26f6429040cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:43.721942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4991d74-546a-4b0d-a4ab-030875dfc53b service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.724848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a4991d74-546a-4b0d-a4ab-030875dfc53b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:43.759914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930457ba-51b4-4dd0-9990-26f6429040cd None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:43.769361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086dbfe-f44d-4c5b-a079-13a98fb9490f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:43.770255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086dbfe-f44d-4c5b-a079-13a98fb9490f service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.770763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086dbfe-f44d-4c5b-a079-13a98fb9490f service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.771333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086dbfe-f44d-4c5b-a079-13a98fb9490f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.771696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4991d74-546a-4b0d-a4ab-030875dfc53b service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:43.774035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0086dbfe-f44d-4c5b-a079-13a98fb9490f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:43.786916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:43.789065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.789351 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.789688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] PATH_INFO: `/v3/projects/196477e33dee45de97c12626bdcdfd55` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.792598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] RBAC: Authorizing `identity:delete_project(project_id=196477e33dee45de97c12626bdcdfd55)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:43.794581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:43.800719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086dbfe-f44d-4c5b-a079-13a98fb9490f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:43.803543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086dbfe-f44d-4c5b-a079-13a98fb9490f service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:43.837832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:43.843400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1230/2430] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:43.873032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '196477e33dee45de97c12626bdcdfd55', 'request_id': 'req-8db08ce6-3500-4d55-ad9a-59d63286b50d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:43.880308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '196477e33dee45de97c12626bdcdfd55', 'request_id': 'req-8db08ce6-3500-4d55-ad9a-59d63286b50d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:43.893049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '196477e33dee45de97c12626bdcdfd55', 'request_id': 'req-8db08ce6-3500-4d55-ad9a-59d63286b50d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:43.907993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] The token cache is being invalidate because project 196477e33dee45de97c12626bdcdfd55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:36:43.908290 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:43.909414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:43.911306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1201/2431] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:36:43 2022] DELETE /identity/v3/projects/196477e33dee45de97c12626bdcdfd55 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:43.915528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086dbfe-f44d-4c5b-a079-13a98fb9490f service nova] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.937704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086dbfe-f44d-4c5b-a079-13a98fb9490f service nova] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:44.023931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:44.025417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.025821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.026366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] PATH_INFO: `/v3/users/ae83bb2bf6404f7ab9b9622d0c433583` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.028951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae83bb2bf6404f7ab9b9622d0c433583)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:44.032458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:44.076041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:44.107638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae83bb2bf6404f7ab9b9622d0c433583'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.126896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae83bb2bf6404f7ab9b9622d0c433583'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.134955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae83bb2bf6404f7ab9b9622d0c433583'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.135659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae83bb2bf6404f7ab9b9622d0c433583'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.136292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-bd7cc643-447a-4d7e-a738-c8201dc0be10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae83bb2bf6404f7ab9b9622d0c433583'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.154644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1231/2432] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:36:43 2022] DELETE /identity/v3/users/ae83bb2bf6404f7ab9b9622d0c433583 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:44.571513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:44.573052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:44.600899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db08ce6-3500-4d55-ad9a-59d63286b50d admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:44.728752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:44.729830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.730215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.730592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] PATH_INFO: `/v3/projects/a80388cecc59479f90cfe6ec59f7b508` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.732158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a80388cecc59479f90cfe6ec59f7b508)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:44.735247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:44.810011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a80388cecc59479f90cfe6ec59f7b508', 'request_id': 'req-d5ba2f78-f105-4a16-8028-7bc89fec8712'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.818685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a80388cecc59479f90cfe6ec59f7b508', 'request_id': 'req-d5ba2f78-f105-4a16-8028-7bc89fec8712'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.829537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a80388cecc59479f90cfe6ec59f7b508', 'request_id': 'req-d5ba2f78-f105-4a16-8028-7bc89fec8712'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.851046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] The token cache is being invalidate because project a80388cecc59479f90cfe6ec59f7b508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:36:44.851478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.852521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d5ba2f78-f105-4a16-8028-7bc89fec8712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:44.855247 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1202/2433] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:36:44 2022] DELETE /identity/v3/projects/a80388cecc59479f90cfe6ec59f7b508 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:44.861795 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751940e-cb5c-4a1e-8893-133754fb9cf7 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.862369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751940e-cb5c-4a1e-8893-133754fb9cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.862754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751940e-cb5c-4a1e-8893-133754fb9cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.881903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d751940e-cb5c-4a1e-8893-133754fb9cf7 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:44.883233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d751940e-cb5c-4a1e-8893-133754fb9cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:44.952857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1232/2434] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:36:44.961597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22495ea-59f2-4be0-b952-2ff699cf7097 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.961902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22495ea-59f2-4be0-b952-2ff699cf7097 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.962245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22495ea-59f2-4be0-b952-2ff699cf7097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.985550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e22495ea-59f2-4be0-b952-2ff699cf7097 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:44.988480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e22495ea-59f2-4be0-b952-2ff699cf7097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:44.997483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1203/2435] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:45.002643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751940e-cb5c-4a1e-8893-133754fb9cf7 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.016421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751940e-cb5c-4a1e-8893-133754fb9cf7 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.021418 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd4257-0463-4b98-8d86-5f509713822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.023096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd4257-0463-4b98-8d86-5f509713822a admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.023380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd4257-0463-4b98-8d86-5f509713822a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.023810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd4257-0463-4b98-8d86-5f509713822a admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.025035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd4257-0463-4b98-8d86-5f509713822a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:45.025507 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd4257-0463-4b98-8d86-5f509713822a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.027263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd4257-0463-4b98-8d86-5f509713822a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.048958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1233/2436] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:45.054002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22495ea-59f2-4be0-b952-2ff699cf7097 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.065780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22495ea-59f2-4be0-b952-2ff699cf7097 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.072025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b805f20-5c15-4058-a802-67be0f45db9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.073253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b805f20-5c15-4058-a802-67be0f45db9f admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.078024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b805f20-5c15-4058-a802-67be0f45db9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.078347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b805f20-5c15-4058-a802-67be0f45db9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.079095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b805f20-5c15-4058-a802-67be0f45db9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.081186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b805f20-5c15-4058-a802-67be0f45db9f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.115425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1204/2437] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:45 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:45.116728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd4257-0463-4b98-8d86-5f509713822a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.129094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd4257-0463-4b98-8d86-5f509713822a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.210818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a9a5-0288-4005-905d-319a2e458018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.211568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a9a5-0288-4005-905d-319a2e458018 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.211880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a9a5-0288-4005-905d-319a2e458018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.212195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a9a5-0288-4005-905d-319a2e458018 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.213037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91a9a5-0288-4005-905d-319a2e458018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.215203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91a9a5-0288-4005-905d-319a2e458018 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.271929 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1234/2438] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:45 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:45.276770 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b805f20-5c15-4058-a802-67be0f45db9f admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.289534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b805f20-5c15-4058-a802-67be0f45db9f admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.294398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa9f76-172d-4552-a8f7-378a9673eda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.295064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa9f76-172d-4552-a8f7-378a9673eda4 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.295531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa9f76-172d-4552-a8f7-378a9673eda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.295813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa9f76-172d-4552-a8f7-378a9673eda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.296557 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa9f76-172d-4552-a8f7-378a9673eda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.298229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa9f76-172d-4552-a8f7-378a9673eda4 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.300330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:45.311235 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1205/2439] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:45.315370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a9a5-0288-4005-905d-319a2e458018 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.325719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a9a5-0288-4005-905d-319a2e458018 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.330809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169fdb0-be45-4d36-9f45-d56e12484d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.331479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169fdb0-be45-4d36-9f45-d56e12484d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.331768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169fdb0-be45-4d36-9f45-d56e12484d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.332049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169fdb0-be45-4d36-9f45-d56e12484d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.332820 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169fdb0-be45-4d36-9f45-d56e12484d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.334460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169fdb0-be45-4d36-9f45-d56e12484d8b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.348480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1235/2440] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:45.349710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa9f76-172d-4552-a8f7-378a9673eda4 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.359773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa9f76-172d-4552-a8f7-378a9673eda4 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.365206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a835a7-d8f0-4da0-b2c9-28f3335e198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.366152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a835a7-d8f0-4da0-b2c9-28f3335e198a admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.366372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a835a7-d8f0-4da0-b2c9-28f3335e198a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.366652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a835a7-d8f0-4da0-b2c9-28f3335e198a admin admin] PATH_INFO: `/v3/projects/348b5f2c788e407facc3c0eb084069cd/users/01d9b845ab2f41119fb22f17b714d280/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.381950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a835a7-d8f0-4da0-b2c9-28f3335e198a admin admin] RBAC: Authorizing `identity:create_grant(project_id=348b5f2c788e407facc3c0eb084069cd, user_id=01d9b845ab2f41119fb22f17b714d280, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.385314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a835a7-d8f0-4da0-b2c9-28f3335e198a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.399358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1206/2441] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:45 2022] PUT /identity/v3/projects/348b5f2c788e407facc3c0eb084069cd/users/01d9b845ab2f41119fb22f17b714d280/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:45.410839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d380e-f969-4e13-adcf-8664a445e889 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.411069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d380e-f969-4e13-adcf-8664a445e889 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.411354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d380e-f969-4e13-adcf-8664a445e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.434681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:45.444789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b26d380e-f969-4e13-adcf-8664a445e889 None None] MFA Rules not processed for user `01d9b845ab2f41119fb22f17b714d280`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:45.446906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b26d380e-f969-4e13-adcf-8664a445e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:45.518739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1236/2442] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:45.523351 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a835a7-d8f0-4da0-b2c9-28f3335e198a admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.598327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a835a7-d8f0-4da0-b2c9-28f3335e198a admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.608072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c096240-6218-4d20-8a13-b45bf0a6ef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.608072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c096240-6218-4d20-8a13-b45bf0a6ef6c admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.608072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c096240-6218-4d20-8a13-b45bf0a6ef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.608072 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c096240-6218-4d20-8a13-b45bf0a6ef6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.608547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c096240-6218-4d20-8a13-b45bf0a6ef6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.608547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c096240-6218-4d20-8a13-b45bf0a6ef6c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.637059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1207/2443] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:45 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:45.641095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d380e-f969-4e13-adcf-8664a445e889 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.655825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d380e-f969-4e13-adcf-8664a445e889 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.717393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef818a1-7cc3-41d6-89e3-c0a7f87e563c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.718080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef818a1-7cc3-41d6-89e3-c0a7f87e563c admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.718403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef818a1-7cc3-41d6-89e3-c0a7f87e563c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.718630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef818a1-7cc3-41d6-89e3-c0a7f87e563c admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.721602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef818a1-7cc3-41d6-89e3-c0a7f87e563c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.722069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef818a1-7cc3-41d6-89e3-c0a7f87e563c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.743395 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1237/2444] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:45 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:45.747487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c096240-6218-4d20-8a13-b45bf0a6ef6c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.758193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c096240-6218-4d20-8a13-b45bf0a6ef6c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.763803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adea6f8-093a-4b98-a682-9e8d91186cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.764674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea6f8-093a-4b98-a682-9e8d91186cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.765056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea6f8-093a-4b98-a682-9e8d91186cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.765441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea6f8-093a-4b98-a682-9e8d91186cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.766414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adea6f8-093a-4b98-a682-9e8d91186cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.768762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adea6f8-093a-4b98-a682-9e8d91186cd4 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.784859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1208/2445] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:36:45.786037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef818a1-7cc3-41d6-89e3-c0a7f87e563c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.796506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef818a1-7cc3-41d6-89e3-c0a7f87e563c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.801781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f634d7-9a28-4b9a-a759-077ec11a1eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.802752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f634d7-9a28-4b9a-a759-077ec11a1eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.803060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f634d7-9a28-4b9a-a759-077ec11a1eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.803319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f634d7-9a28-4b9a-a759-077ec11a1eb2 admin admin] PATH_INFO: `/v3/projects/bdfd62f00729412ca1170ac29e8ec8af/users/d8daa062bf594cc2810b09e6c9c0a2be/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.820284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f634d7-9a28-4b9a-a759-077ec11a1eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfd62f00729412ca1170ac29e8ec8af, user_id=d8daa062bf594cc2810b09e6c9c0a2be, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.823248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f634d7-9a28-4b9a-a759-077ec11a1eb2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.838132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1238/2446] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:45 2022] PUT /identity/v3/projects/bdfd62f00729412ca1170ac29e8ec8af/users/d8daa062bf594cc2810b09e6c9c0a2be/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:45.842122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adea6f8-093a-4b98-a682-9e8d91186cd4 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.853108 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adea6f8-093a-4b98-a682-9e8d91186cd4 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.908470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2039ca61-df0d-4d40-b44c-1dbcbf05ed41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.909234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039ca61-df0d-4d40-b44c-1dbcbf05ed41 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.909512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039ca61-df0d-4d40-b44c-1dbcbf05ed41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.909785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039ca61-df0d-4d40-b44c-1dbcbf05ed41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.910513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039ca61-df0d-4d40-b44c-1dbcbf05ed41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.912162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039ca61-df0d-4d40-b44c-1dbcbf05ed41 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.921942 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1209/2447] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:45.926296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f634d7-9a28-4b9a-a759-077ec11a1eb2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.936487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f634d7-9a28-4b9a-a759-077ec11a1eb2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.941091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f99d3-eac5-4162-b495-5e4825d5a3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.942005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f99d3-eac5-4162-b495-5e4825d5a3af admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.942275 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f99d3-eac5-4162-b495-5e4825d5a3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.942667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f99d3-eac5-4162-b495-5e4825d5a3af admin admin] PATH_INFO: `/v3/projects/bdfd62f00729412ca1170ac29e8ec8af/users/d8daa062bf594cc2810b09e6c9c0a2be/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.944725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f99d3-eac5-4162-b495-5e4825d5a3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfd62f00729412ca1170ac29e8ec8af, user_id=d8daa062bf594cc2810b09e6c9c0a2be, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.947374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f99d3-eac5-4162-b495-5e4825d5a3af admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.958276 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1239/2448] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:45 2022] PUT /identity/v3/projects/bdfd62f00729412ca1170ac29e8ec8af/users/d8daa062bf594cc2810b09e6c9c0a2be/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:45.968330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817705d-365e-4181-a7cf-abf1676118a7 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.968621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817705d-365e-4181-a7cf-abf1676118a7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.968929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817705d-365e-4181-a7cf-abf1676118a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.995066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a817705d-365e-4181-a7cf-abf1676118a7 None None] MFA Rules not processed for user `d8daa062bf594cc2810b09e6c9c0a2be`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:45.996254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a817705d-365e-4181-a7cf-abf1676118a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:46.026698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:46.067733 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1210/2449] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:46.078736 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f99d3-eac5-4162-b495-5e4825d5a3af admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:46.081367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fc1f99d3-eac5-4162-b495-5e4825d5a3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:46.140201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f99d3-eac5-4162-b495-5e4825d5a3af admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:46.145268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba362d0-715b-4dea-88b2-d3ecbd5588de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:46.145916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba362d0-715b-4dea-88b2-d3ecbd5588de service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:46.146199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba362d0-715b-4dea-88b2-d3ecbd5588de service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:46.146479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba362d0-715b-4dea-88b2-d3ecbd5588de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:46.151961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:46.203185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba362d0-715b-4dea-88b2-d3ecbd5588de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:46.204559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba362d0-715b-4dea-88b2-d3ecbd5588de service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:46.213931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1240/2450] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:48.876057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817705d-365e-4181-a7cf-abf1676118a7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:48.877796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a817705d-365e-4181-a7cf-abf1676118a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:48.951488 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817705d-365e-4181-a7cf-abf1676118a7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:48.955632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b3836-82aa-47d0-ae8d-407b06dc8fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:48.956313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b3836-82aa-47d0-ae8d-407b06dc8fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:48.956598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b3836-82aa-47d0-ae8d-407b06dc8fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:48.956892 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b3836-82aa-47d0-ae8d-407b06dc8fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:48.958706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f61b3836-82aa-47d0-ae8d-407b06dc8fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:49.023109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b3836-82aa-47d0-ae8d-407b06dc8fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:49.024860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b3836-82aa-47d0-ae8d-407b06dc8fa7 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:49.035628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1211/2451] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:49.296545 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba362d0-715b-4dea-88b2-d3ecbd5588de service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:49.299414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-aba362d0-715b-4dea-88b2-d3ecbd5588de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:49.316238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba362d0-715b-4dea-88b2-d3ecbd5588de service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:49.321109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:49.322299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:49.322756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:49.323182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:49.325818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:49.349039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:49.350863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:49.362865 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1241/2452] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:50.462342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b3836-82aa-47d0-ae8d-407b06dc8fa7 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:50.473740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b3836-82aa-47d0-ae8d-407b06dc8fa7 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:50.477663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296c767-7d44-4812-8635-9066cad556d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:50.478589 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296c767-7d44-4812-8635-9066cad556d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.479043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296c767-7d44-4812-8635-9066cad556d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.479474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296c767-7d44-4812-8635-9066cad556d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.490798 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296c767-7d44-4812-8635-9066cad556d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:50.492469 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296c767-7d44-4812-8635-9066cad556d7 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:50.504834 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1212/2453] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:53.272324 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:53.273923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:53.353634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296c767-7d44-4812-8635-9066cad556d7 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:53.355211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5296c767-7d44-4812-8635-9066cad556d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:53.390680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296c767-7d44-4812-8635-9066cad556d7 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:53.412386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:53.417246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:53.417541 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:53.417844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] PATH_INFO: `/v3/users/06431315dc8040cbbafbe83a3ed8e0e4` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:53.421275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06431315dc8040cbbafbe83a3ed8e0e4)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:53.424566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:53.433481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e6af-ee0f-4b6f-8866-e9d01b2e1cde service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:53.444749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99e0c1-0b42-4056-8ec3-c991c5e4c965 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:53.445573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e0c1-0b42-4056-8ec3-c991c5e4c965 service placement] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:53.445925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e0c1-0b42-4056-8ec3-c991c5e4c965 service placement] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:53.446128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e0c1-0b42-4056-8ec3-c991c5e4c965 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:53.448598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8c99e0c1-0b42-4056-8ec3-c991c5e4c965 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:53.478539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99e0c1-0b42-4056-8ec3-c991c5e4c965 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:53.480367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99e0c1-0b42-4056-8ec3-c991c5e4c965 service placement] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:53.504700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1242/2454] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:53.550074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06431315dc8040cbbafbe83a3ed8e0e4'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:53.551022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06431315dc8040cbbafbe83a3ed8e0e4'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:53.559976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06431315dc8040cbbafbe83a3ed8e0e4'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:53.560644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06431315dc8040cbbafbe83a3ed8e0e4'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:53.563486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:53.576732 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06431315dc8040cbbafbe83a3ed8e0e4'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:53.592712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1213/2455] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:36:53 2022] DELETE /identity/v3/users/06431315dc8040cbbafbe83a3ed8e0e4 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:53.857778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99e0c1-0b42-4056-8ec3-c991c5e4c965 service placement] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:53.870303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99e0c1-0b42-4056-8ec3-c991c5e4c965 service placement] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:53.945753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:53.947728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:53.950012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:53.950883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:53.951176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:53.951454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] PATH_INFO: `/v3/projects/f2020e2bf35744af9161d7cf3e8bd345` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:53.952660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2020e2bf35744af9161d7cf3e8bd345)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:53.954613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:53.990109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.014117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2020e2bf35744af9161d7cf3e8bd345', 'request_id': 'req-ed53290a-82fe-42e6-aca7-21a37be0ed49'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.021461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2020e2bf35744af9161d7cf3e8bd345', 'request_id': 'req-ed53290a-82fe-42e6-aca7-21a37be0ed49'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.028608 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2020e2bf35744af9161d7cf3e8bd345', 'request_id': 'req-ed53290a-82fe-42e6-aca7-21a37be0ed49'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.038901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c730efb-a30c-446d-8aa8-533a72777877 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.043544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] The token cache is being invalidate because project f2020e2bf35744af9161d7cf3e8bd345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:36:54.043836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.044468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.046316 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1243/2456] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:36:53 2022] DELETE /identity/v3/projects/f2020e2bf35744af9161d7cf3e8bd345 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:54.046862 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.048161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.115828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53290a-82fe-42e6-aca7-21a37be0ed49 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.116447 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c262e19-f187-42e7-a16b-7a7515dcfe3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.117216 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c262e19-f187-42e7-a16b-7a7515dcfe3c service placement] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.117683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c262e19-f187-42e7-a16b-7a7515dcfe3c service placement] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.118177 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c262e19-f187-42e7-a16b-7a7515dcfe3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.120298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6c262e19-f187-42e7-a16b-7a7515dcfe3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.120827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.121724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.122008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.122283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] PATH_INFO: `/v3/users/d0ae34a029c14c6f826456414ac0670d` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.123472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ae34a029c14c6f826456414ac0670d)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.125409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.132506 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c262e19-f187-42e7-a16b-7a7515dcfe3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.134139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c262e19-f187-42e7-a16b-7a7515dcfe3c service placement] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.144582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1214/2457] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:36:54.149333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069068c-431c-4b19-a48d-48b9978cd3a6 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.149691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069068c-431c-4b19-a48d-48b9978cd3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.150233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069068c-431c-4b19-a48d-48b9978cd3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.174788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4069068c-431c-4b19-a48d-48b9978cd3a6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:54.176694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4069068c-431c-4b19-a48d-48b9978cd3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.177479 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ae34a029c14c6f826456414ac0670d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.185208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1215/2458] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:54.191300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ae34a029c14c6f826456414ac0670d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.193702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f093ff-e9c2-4609-bf5b-cfce1ec233bf None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.194434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f093ff-e9c2-4609-bf5b-cfce1ec233bf None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.194724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f093ff-e9c2-4609-bf5b-cfce1ec233bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.200922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ae34a029c14c6f826456414ac0670d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.201738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ae34a029c14c6f826456414ac0670d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.202445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ae34a029c14c6f826456414ac0670d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.215128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-76f093ff-e9c2-4609-bf5b-cfce1ec233bf None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:54.216337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-76f093ff-e9c2-4609-bf5b-cfce1ec233bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.218235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1244/2459] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:36:54 2022] DELETE /identity/v3/users/d0ae34a029c14c6f826456414ac0670d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:54.290729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1216/2460] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:36:54.295293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.306657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7ef3b-57cb-4904-81b5-b74b4694c3cd admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.311654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a7252-170f-4e53-9e0a-cc993bb61642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.312538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a7252-170f-4e53-9e0a-cc993bb61642 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.312829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a7252-170f-4e53-9e0a-cc993bb61642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.313148 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a7252-170f-4e53-9e0a-cc993bb61642 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.313895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a7252-170f-4e53-9e0a-cc993bb61642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:54.314719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a7252-170f-4e53-9e0a-cc993bb61642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.316543 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a7252-170f-4e53-9e0a-cc993bb61642 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.328772 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1245/2461] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:54.333957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f093ff-e9c2-4609-bf5b-cfce1ec233bf None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.338504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.344133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f093ff-e9c2-4609-bf5b-cfce1ec233bf None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.351693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a82f33-4696-4af5-ace6-ff4d8d6e3749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.352584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a82f33-4696-4af5-ace6-ff4d8d6e3749 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.353023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a82f33-4696-4af5-ace6-ff4d8d6e3749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.353468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a82f33-4696-4af5-ace6-ff4d8d6e3749 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.354752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a82f33-4696-4af5-ace6-ff4d8d6e3749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.356832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a82f33-4696-4af5-ace6-ff4d8d6e3749 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.391007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1217/2462] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/projects => generated 357 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:54.395848 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a7252-170f-4e53-9e0a-cc993bb61642 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.415640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a82f33-4696-4af5-ace6-ff4d8d6e3749 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.417227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a7252-170f-4e53-9e0a-cc993bb61642 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.417375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d5a82f33-4696-4af5-ace6-ff4d8d6e3749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.462167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.489090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a82f33-4696-4af5-ace6-ff4d8d6e3749 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.493810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.494676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.494963 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.495236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] PATH_INFO: `/v3/projects/9aaff9db173d4618929536b08d2be8f0` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.496420 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aaff9db173d4618929536b08d2be8f0)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.498415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.504206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e7c15-8cf9-49f0-a19c-cc92ca56c335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.504861 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7c15-8cf9-49f0-a19c-cc92ca56c335 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.505176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7c15-8cf9-49f0-a19c-cc92ca56c335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.505446 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7c15-8cf9-49f0-a19c-cc92ca56c335 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.506311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e7c15-8cf9-49f0-a19c-cc92ca56c335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.508248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e7c15-8cf9-49f0-a19c-cc92ca56c335 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.530487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1246/2463] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/users => generated 279 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:54.534435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e7c15-8cf9-49f0-a19c-cc92ca56c335 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.544478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e7c15-8cf9-49f0-a19c-cc92ca56c335 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.551013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aaff9db173d4618929536b08d2be8f0', 'request_id': 'req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.559839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aaff9db173d4618929536b08d2be8f0', 'request_id': 'req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.572617 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aaff9db173d4618929536b08d2be8f0', 'request_id': 'req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.598968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] The token cache is being invalidate because project 9aaff9db173d4618929536b08d2be8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:36:54.599268 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.599950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-394a9ac2-7495-4be9-bdc7-d6d5750c7cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:54.604956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1218/2464] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:36:54 2022] DELETE /identity/v3/projects/9aaff9db173d4618929536b08d2be8f0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:54.607145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab8f77-e1b6-454a-bf5d-6077cd90f976 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.607437 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab8f77-e1b6-454a-bf5d-6077cd90f976 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.607757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab8f77-e1b6-454a-bf5d-6077cd90f976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.632168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744cc4e-892d-4872-b4c5-f6838dee27c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.632980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744cc4e-892d-4872-b4c5-f6838dee27c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.633353 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744cc4e-892d-4872-b4c5-f6838dee27c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.633697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744cc4e-892d-4872-b4c5-f6838dee27c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.635099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744cc4e-892d-4872-b4c5-f6838dee27c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.635371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-95ab8f77-e1b6-454a-bf5d-6077cd90f976 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:54.636427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744cc4e-892d-4872-b4c5-f6838dee27c9 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.637152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-95ab8f77-e1b6-454a-bf5d-6077cd90f976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.647531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1247/2465] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:36:54.651589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744cc4e-892d-4872-b4c5-f6838dee27c9 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.653314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6744cc4e-892d-4872-b4c5-f6838dee27c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.660445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1219/2466] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:54.663207 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170e14-1618-4c77-9ea4-2376b2c047c1 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.664031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170e14-1618-4c77-9ea4-2376b2c047c1 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.664423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170e14-1618-4c77-9ea4-2376b2c047c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.666094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744cc4e-892d-4872-b4c5-f6838dee27c9 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.671120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09590fde-11cd-4823-afea-1fd82e0d0bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.672028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09590fde-11cd-4823-afea-1fd82e0d0bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.672311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09590fde-11cd-4823-afea-1fd82e0d0bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.672746 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09590fde-11cd-4823-afea-1fd82e0d0bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.674255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09590fde-11cd-4823-afea-1fd82e0d0bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.676981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09590fde-11cd-4823-afea-1fd82e0d0bcd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.688989 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1248/2467] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:36:54.692523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09590fde-11cd-4823-afea-1fd82e0d0bcd admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.697341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e3170e14-1618-4c77-9ea4-2376b2c047c1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:54.698565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e3170e14-1618-4c77-9ea4-2376b2c047c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.702658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09590fde-11cd-4823-afea-1fd82e0d0bcd admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.710611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1220/2468] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:54.710785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c6203-cc66-427b-8e8f-7a8ce3c8001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.710785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c6203-cc66-427b-8e8f-7a8ce3c8001f admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.710785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c6203-cc66-427b-8e8f-7a8ce3c8001f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.710785 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c6203-cc66-427b-8e8f-7a8ce3c8001f admin admin] PATH_INFO: `/v3/projects/ecb3a604bce84d20aa08c32faa4b4013/users/f35664e135bb4c338a46332f9e29343d/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.711621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170e14-1618-4c77-9ea4-2376b2c047c1 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.722094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170e14-1618-4c77-9ea4-2376b2c047c1 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.725314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c6203-cc66-427b-8e8f-7a8ce3c8001f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb3a604bce84d20aa08c32faa4b4013, user_id=f35664e135bb4c338a46332f9e29343d, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.727483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6bc61-2938-44a1-bad8-028d7d2e4e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.728023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c6203-cc66-427b-8e8f-7a8ce3c8001f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.728285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6bc61-2938-44a1-bad8-028d7d2e4e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.728627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6bc61-2938-44a1-bad8-028d7d2e4e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.729016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6bc61-2938-44a1-bad8-028d7d2e4e18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.729805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6bc61-2938-44a1-bad8-028d7d2e4e18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:54.730176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6bc61-2938-44a1-bad8-028d7d2e4e18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.731829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6bc61-2938-44a1-bad8-028d7d2e4e18 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.752744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1249/2469] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:54 2022] PUT /identity/v3/projects/ecb3a604bce84d20aa08c32faa4b4013/users/f35664e135bb4c338a46332f9e29343d/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:54.752976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1221/2470] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:54.753113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c6203-cc66-427b-8e8f-7a8ce3c8001f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.754366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b679b2-d87c-4e9a-b916-a09b5e2ea00f None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.754711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b679b2-d87c-4e9a-b916-a09b5e2ea00f None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.754989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b679b2-d87c-4e9a-b916-a09b5e2ea00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.782250 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a5b679b2-d87c-4e9a-b916-a09b5e2ea00f None None] MFA Rules not processed for user `f35664e135bb4c338a46332f9e29343d`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:54.784409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a5b679b2-d87c-4e9a-b916-a09b5e2ea00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.816930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c6203-cc66-427b-8e8f-7a8ce3c8001f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.817821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.821954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde97e1c-0b20-4fa9-b381-0ec0364fe6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.822665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde97e1c-0b20-4fa9-b381-0ec0364fe6cb admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.822951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde97e1c-0b20-4fa9-b381-0ec0364fe6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.823380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde97e1c-0b20-4fa9-b381-0ec0364fe6cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.824224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde97e1c-0b20-4fa9-b381-0ec0364fe6cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.826280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde97e1c-0b20-4fa9-b381-0ec0364fe6cb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.841216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.860454 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1250/2471] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:54.860694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1222/2472] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:54.861573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde97e1c-0b20-4fa9-b381-0ec0364fe6cb admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.864475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b679b2-d87c-4e9a-b916-a09b5e2ea00f None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.875025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b679b2-d87c-4e9a-b916-a09b5e2ea00f None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.933639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde97e1c-0b20-4fa9-b381-0ec0364fe6cb admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.939022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d1228-78a9-42b6-8f93-110393ea6fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.939789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d1228-78a9-42b6-8f93-110393ea6fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.940070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d1228-78a9-42b6-8f93-110393ea6fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.940388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d1228-78a9-42b6-8f93-110393ea6fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.940689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d82f3-b306-41fe-b7eb-d450a7c5db37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.941156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d1228-78a9-42b6-8f93-110393ea6fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.941648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d82f3-b306-41fe-b7eb-d450a7c5db37 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.942083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d82f3-b306-41fe-b7eb-d450a7c5db37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.942502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d82f3-b306-41fe-b7eb-d450a7c5db37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.943208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d1228-78a9-42b6-8f93-110393ea6fb3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.943324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d82f3-b306-41fe-b7eb-d450a7c5db37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.945375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d82f3-b306-41fe-b7eb-d450a7c5db37 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.968208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1251/2473] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:54.973345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d1228-78a9-42b6-8f93-110393ea6fb3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.982374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1223/2474] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/projects => generated 357 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:54.983662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d82f3-b306-41fe-b7eb-d450a7c5db37 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.983799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d1228-78a9-42b6-8f93-110393ea6fb3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.051774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbdbfa-9b4b-4e19-9f76-e1487e79c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.052594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbdbfa-9b4b-4e19-9f76-e1487e79c1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.053192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbdbfa-9b4b-4e19-9f76-e1487e79c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.053326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbdbfa-9b4b-4e19-9f76-e1487e79c1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.054137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbdbfa-9b4b-4e19-9f76-e1487e79c1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.055988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbdbfa-9b4b-4e19-9f76-e1487e79c1a5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.064676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d82f3-b306-41fe-b7eb-d450a7c5db37 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.071679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1252/2475] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:36:55.072710 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbdbfa-9b4b-4e19-9f76-e1487e79c1a5 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.073624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea8480-9c00-4a17-9309-ce5583482575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.074432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea8480-9c00-4a17-9309-ce5583482575 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.074822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea8480-9c00-4a17-9309-ce5583482575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.075202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea8480-9c00-4a17-9309-ce5583482575 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.076051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea8480-9c00-4a17-9309-ce5583482575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.078262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea8480-9c00-4a17-9309-ce5583482575 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.085270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbdbfa-9b4b-4e19-9f76-e1487e79c1a5 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.091971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45e5b9-d282-43b7-9112-878d27fc32de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.092739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45e5b9-d282-43b7-9112-878d27fc32de admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.093172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45e5b9-d282-43b7-9112-878d27fc32de admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.093456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45e5b9-d282-43b7-9112-878d27fc32de admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.094392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45e5b9-d282-43b7-9112-878d27fc32de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.096520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45e5b9-d282-43b7-9112-878d27fc32de admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.104655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1224/2476] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/users => generated 285 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:55.108912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1253/2477] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:36:55.109070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea8480-9c00-4a17-9309-ce5583482575 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.111736 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45e5b9-d282-43b7-9112-878d27fc32de admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.120399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea8480-9c00-4a17-9309-ce5583482575 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.123255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45e5b9-d282-43b7-9112-878d27fc32de admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.125431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b65d6f-4329-4cbb-b999-70d0db60be71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.126112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b65d6f-4329-4cbb-b999-70d0db60be71 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.126404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b65d6f-4329-4cbb-b999-70d0db60be71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.127215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b65d6f-4329-4cbb-b999-70d0db60be71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.127964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b65d6f-4329-4cbb-b999-70d0db60be71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.129627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b65d6f-4329-4cbb-b999-70d0db60be71 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.130076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b601c-1be9-4f2b-b649-f8388031bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.131420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b601c-1be9-4f2b-b649-f8388031bb7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.131886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b601c-1be9-4f2b-b649-f8388031bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.132359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b601c-1be9-4f2b-b649-f8388031bb7c admin admin] PATH_INFO: `/v3/projects/7b5d304bbc7742a5a27808981fd17ad2/users/80870555a9a84101a7fade0e629c0b49/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.141789 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1225/2478] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:36:55.150229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b65d6f-4329-4cbb-b999-70d0db60be71 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.151968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b601c-1be9-4f2b-b649-f8388031bb7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5d304bbc7742a5a27808981fd17ad2, user_id=80870555a9a84101a7fade0e629c0b49, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.154755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b601c-1be9-4f2b-b649-f8388031bb7c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.161110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b65d6f-4329-4cbb-b999-70d0db60be71 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.166307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce759b-832b-4eef-af4f-32e361cb3b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.167206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce759b-832b-4eef-af4f-32e361cb3b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.167486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce759b-832b-4eef-af4f-32e361cb3b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.167756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce759b-832b-4eef-af4f-32e361cb3b06 admin admin] PATH_INFO: `/v3/projects/b9829182b3e44100b8bac79a58a650d5/users/fb31f4f301d448f8a2d0974306166fc8/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.169552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1254/2479] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:55 2022] PUT /identity/v3/projects/7b5d304bbc7742a5a27808981fd17ad2/users/80870555a9a84101a7fade0e629c0b49/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:55.180452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76970db-c409-4dbc-a4d9-9f8a8b659a80 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.180734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76970db-c409-4dbc-a4d9-9f8a8b659a80 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.181046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76970db-c409-4dbc-a4d9-9f8a8b659a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.186284 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce759b-832b-4eef-af4f-32e361cb3b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9829182b3e44100b8bac79a58a650d5, user_id=fb31f4f301d448f8a2d0974306166fc8, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.188885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce759b-832b-4eef-af4f-32e361cb3b06 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.205846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1226/2480] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:55 2022] PUT /identity/v3/projects/b9829182b3e44100b8bac79a58a650d5/users/fb31f4f301d448f8a2d0974306166fc8/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:55.210497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce759b-832b-4eef-af4f-32e361cb3b06 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.210673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b76970db-c409-4dbc-a4d9-9f8a8b659a80 None None] MFA Rules not processed for user `80870555a9a84101a7fade0e629c0b49`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:55.212572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b76970db-c409-4dbc-a4d9-9f8a8b659a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:55.226680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.247184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.278467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce759b-832b-4eef-af4f-32e361cb3b06 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.281441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1255/2481] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:36:55.283374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0325393-f45a-47a1-afd5-f16bf1481239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.284279 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0325393-f45a-47a1-afd5-f16bf1481239 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.284396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0325393-f45a-47a1-afd5-f16bf1481239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.284647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0325393-f45a-47a1-afd5-f16bf1481239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.285418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0325393-f45a-47a1-afd5-f16bf1481239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.287347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0325393-f45a-47a1-afd5-f16bf1481239 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.292137 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76970db-c409-4dbc-a4d9-9f8a8b659a80 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.293641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b76970db-c409-4dbc-a4d9-9f8a8b659a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:55.297914 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1227/2482] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:36:55.301850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0325393-f45a-47a1-afd5-f16bf1481239 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.311225 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0325393-f45a-47a1-afd5-f16bf1481239 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.316320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca4bc5-84fb-4b00-8b14-4aadb0a31253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.317204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca4bc5-84fb-4b00-8b14-4aadb0a31253 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.317480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca4bc5-84fb-4b00-8b14-4aadb0a31253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.317747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca4bc5-84fb-4b00-8b14-4aadb0a31253 admin admin] PATH_INFO: `/v3/projects/b9829182b3e44100b8bac79a58a650d5/users/fb31f4f301d448f8a2d0974306166fc8/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.319676 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca4bc5-84fb-4b00-8b14-4aadb0a31253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9829182b3e44100b8bac79a58a650d5, user_id=fb31f4f301d448f8a2d0974306166fc8, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.322349 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca4bc5-84fb-4b00-8b14-4aadb0a31253 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.334551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1228/2483] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:55 2022] PUT /identity/v3/projects/b9829182b3e44100b8bac79a58a650d5/users/fb31f4f301d448f8a2d0974306166fc8/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:55.346019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b30e5-28d4-40e4-973e-156d5628ee69 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.346339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b30e5-28d4-40e4-973e-156d5628ee69 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.346625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b30e5-28d4-40e4-973e-156d5628ee69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.370614 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76970db-c409-4dbc-a4d9-9f8a8b659a80 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.374020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-ff8b30e5-28d4-40e4-973e-156d5628ee69 None None] MFA Rules not processed for user `fb31f4f301d448f8a2d0974306166fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:55.375263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ff8b30e5-28d4-40e4-973e-156d5628ee69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:55.441441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9049c6-2532-4c09-b7db-b60a3da8ecb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.442055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9049c6-2532-4c09-b7db-b60a3da8ecb4 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.442340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9049c6-2532-4c09-b7db-b60a3da8ecb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.442620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9049c6-2532-4c09-b7db-b60a3da8ecb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.457090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1229/2484] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:36:55.469608 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b30e5-28d4-40e4-973e-156d5628ee69 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.471082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ff8b30e5-28d4-40e4-973e-156d5628ee69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:55.506252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9049c6-2532-4c09-b7db-b60a3da8ecb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.508094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9049c6-2532-4c09-b7db-b60a3da8ecb4 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.518199 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1256/2485] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:36:55.531636 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b30e5-28d4-40e4-973e-156d5628ee69 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.535583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548df2d-3199-47d7-86bb-0421a935d4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.536552 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548df2d-3199-47d7-86bb-0421a935d4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.536845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548df2d-3199-47d7-86bb-0421a935d4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.537220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548df2d-3199-47d7-86bb-0421a935d4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.551175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548df2d-3199-47d7-86bb-0421a935d4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.553543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548df2d-3199-47d7-86bb-0421a935d4b6 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.568805 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1230/2486] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:36:55.654905 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9049c6-2532-4c09-b7db-b60a3da8ecb4 service nova] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.665038 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9049c6-2532-4c09-b7db-b60a3da8ecb4 service nova] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.668687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a0587-6c1a-4ad5-bdc5-d0331d41fd1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.669357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a0587-6c1a-4ad5-bdc5-d0331d41fd1a service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.669646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a0587-6c1a-4ad5-bdc5-d0331d41fd1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.669927 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a0587-6c1a-4ad5-bdc5-d0331d41fd1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.679132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.734540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a0587-6c1a-4ad5-bdc5-d0331d41fd1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.736126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a0587-6c1a-4ad5-bdc5-d0331d41fd1a service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.745607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1257/2487] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:55.813762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548df2d-3199-47d7-86bb-0421a935d4b6 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.815811 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f548df2d-3199-47d7-86bb-0421a935d4b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:55.866893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.929930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548df2d-3199-47d7-86bb-0421a935d4b6 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.943168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.944744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.945284 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.945840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] PATH_INFO: `/v3/users/01d9b845ab2f41119fb22f17b714d280` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.948059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d9b845ab2f41119fb22f17b714d280)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.950863 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.998007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d9b845ab2f41119fb22f17b714d280'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:55.998641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d9b845ab2f41119fb22f17b714d280'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.005523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d9b845ab2f41119fb22f17b714d280'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.006156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d9b845ab2f41119fb22f17b714d280'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.017827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d9b845ab2f41119fb22f17b714d280'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.031436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1231/2488] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:36:55 2022] DELETE /identity/v3/users/01d9b845ab2f41119fb22f17b714d280 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:56.037037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678f8e-d995-4937-ac2f-7b0f487e41ad None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.037329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678f8e-d995-4937-ac2f-7b0f487e41ad None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.037608 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678f8e-d995-4937-ac2f-7b0f487e41ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.055375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4b678f8e-d995-4937-ac2f-7b0f487e41ad None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:56.056757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4b678f8e-d995-4937-ac2f-7b0f487e41ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:56.115382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1258/2489] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:56.125101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.126489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:56.186182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d91a2-be13-4679-9750-0532adaa82d6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.189966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d913020-85f7-4ab0-b635-6aa9de722987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.190602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d913020-85f7-4ab0-b635-6aa9de722987 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.190886 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d913020-85f7-4ab0-b635-6aa9de722987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.191163 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d913020-85f7-4ab0-b635-6aa9de722987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.200042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d913020-85f7-4ab0-b635-6aa9de722987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:56.201521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d913020-85f7-4ab0-b635-6aa9de722987 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:56.211402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1232/2490] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:56.477682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b678f8e-d995-4937-ac2f-7b0f487e41ad None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.479183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4b678f8e-d995-4937-ac2f-7b0f487e41ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:56.491437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b678f8e-d995-4937-ac2f-7b0f487e41ad None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.496102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.497006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.497291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.497621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] PATH_INFO: `/v3/projects/348b5f2c788e407facc3c0eb084069cd` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.498786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=348b5f2c788e407facc3c0eb084069cd)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:56.500701 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:56.549399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '348b5f2c788e407facc3c0eb084069cd', 'request_id': 'req-466affa7-4361-4632-8511-a549d14f71b3'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.555503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '348b5f2c788e407facc3c0eb084069cd', 'request_id': 'req-466affa7-4361-4632-8511-a549d14f71b3'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.561008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '348b5f2c788e407facc3c0eb084069cd', 'request_id': 'req-466affa7-4361-4632-8511-a549d14f71b3'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.573563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] The token cache is being invalidate because project 348b5f2c788e407facc3c0eb084069cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:36:56.573901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.574492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.576020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1259/2491] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:36:56 2022] DELETE /identity/v3/projects/348b5f2c788e407facc3c0eb084069cd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:56.579970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d913020-85f7-4ab0-b635-6aa9de722987 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.590704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d913020-85f7-4ab0-b635-6aa9de722987 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.633969 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:56.648735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.649655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.649935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.650465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] PATH_INFO: `/v3/users/d8daa062bf594cc2810b09e6c9c0a2be` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.651717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8daa062bf594cc2810b09e6c9c0a2be)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:56.653687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:56.699330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8daa062bf594cc2810b09e6c9c0a2be'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.699986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8daa062bf594cc2810b09e6c9c0a2be'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.708079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8daa062bf594cc2810b09e6c9c0a2be'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.708740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8daa062bf594cc2810b09e6c9c0a2be'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.720510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-c6217e3a-669c-4a6e-b814-3066fd694649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8daa062bf594cc2810b09e6c9c0a2be'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:56.733172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1233/2492] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:36:56 2022] DELETE /identity/v3/users/d8daa062bf594cc2810b09e6c9c0a2be => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:57.014520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.016368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:57.048425 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466affa7-4361-4632-8511-a549d14f71b3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.092926 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:57.141878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.143872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.144233 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.144603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] PATH_INFO: `/v3/projects/bdfd62f00729412ca1170ac29e8ec8af` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.146215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdfd62f00729412ca1170ac29e8ec8af)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.148155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.203954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdfd62f00729412ca1170ac29e8ec8af', 'request_id': 'req-300dd6be-dc39-41b2-afab-1431ea3b8015'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:57.210642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdfd62f00729412ca1170ac29e8ec8af', 'request_id': 'req-300dd6be-dc39-41b2-afab-1431ea3b8015'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:57.216049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdfd62f00729412ca1170ac29e8ec8af', 'request_id': 'req-300dd6be-dc39-41b2-afab-1431ea3b8015'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:57.230693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] The token cache is being invalidate because project bdfd62f00729412ca1170ac29e8ec8af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:36:57.231057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:57.231742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-300dd6be-dc39-41b2-afab-1431ea3b8015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:57.233501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1260/2493] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:36:57 2022] DELETE /identity/v3/projects/bdfd62f00729412ca1170ac29e8ec8af => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:57.239564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95f779-1208-4973-b65e-541cbad87cbd None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.239851 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95f779-1208-4973-b65e-541cbad87cbd None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.240126 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95f779-1208-4973-b65e-541cbad87cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.259814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-2b95f779-1208-4973-b65e-541cbad87cbd None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:57.261031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-2b95f779-1208-4973-b65e-541cbad87cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:57.319583 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1234/2494] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:57.328049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea2386-d502-4c98-a61c-1e93eb2aa1df None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.328342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea2386-d502-4c98-a61c-1e93eb2aa1df None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.328626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea2386-d502-4c98-a61c-1e93eb2aa1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.347931 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-16ea2386-d502-4c98-a61c-1e93eb2aa1df None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:57.349116 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-16ea2386-d502-4c98-a61c-1e93eb2aa1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:57.356847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1261/2495] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:57.361042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95f779-1208-4973-b65e-541cbad87cbd None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.371114 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95f779-1208-4973-b65e-541cbad87cbd None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.375795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afdba7-b4c6-4af6-b6d5-048b854ee984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.376499 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afdba7-b4c6-4af6-b6d5-048b854ee984 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.377382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afdba7-b4c6-4af6-b6d5-048b854ee984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.377660 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afdba7-b4c6-4af6-b6d5-048b854ee984 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.378389 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afdba7-b4c6-4af6-b6d5-048b854ee984 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:57.378698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afdba7-b4c6-4af6-b6d5-048b854ee984 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.380423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afdba7-b4c6-4af6-b6d5-048b854ee984 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.393302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1235/2496] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:57.397786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea2386-d502-4c98-a61c-1e93eb2aa1df None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.408048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea2386-d502-4c98-a61c-1e93eb2aa1df None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.412760 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664ab70-a23b-445d-925c-acb6029fef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.413450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664ab70-a23b-445d-925c-acb6029fef22 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.413733 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664ab70-a23b-445d-925c-acb6029fef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.414035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664ab70-a23b-445d-925c-acb6029fef22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.414766 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664ab70-a23b-445d-925c-acb6029fef22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.416797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664ab70-a23b-445d-925c-acb6029fef22 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.441631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1262/2497] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:57 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:57.446146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afdba7-b4c6-4af6-b6d5-048b854ee984 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.457335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afdba7-b4c6-4af6-b6d5-048b854ee984 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.506831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:57.518449 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ba63-7857-46fd-8c04-e60d354a5c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.519402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ba63-7857-46fd-8c04-e60d354a5c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.519644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ba63-7857-46fd-8c04-e60d354a5c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.519918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ba63-7857-46fd-8c04-e60d354a5c32 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.520625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478ba63-7857-46fd-8c04-e60d354a5c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.522755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478ba63-7857-46fd-8c04-e60d354a5c32 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.542314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1236/2498] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:57 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:57.546590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664ab70-a23b-445d-925c-acb6029fef22 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.556981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664ab70-a23b-445d-925c-acb6029fef22 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.561537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a354f6-21c1-42ef-8d33-247a64f36dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.562193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a354f6-21c1-42ef-8d33-247a64f36dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.562476 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a354f6-21c1-42ef-8d33-247a64f36dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.562755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a354f6-21c1-42ef-8d33-247a64f36dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.563498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a354f6-21c1-42ef-8d33-247a64f36dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.565786 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a354f6-21c1-42ef-8d33-247a64f36dfe admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.576094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1263/2499] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:57.580539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ba63-7857-46fd-8c04-e60d354a5c32 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.590553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ba63-7857-46fd-8c04-e60d354a5c32 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.595150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5742f-929b-4173-b98d-598dcc663532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.595908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5742f-929b-4173-b98d-598dcc663532 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.596193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5742f-929b-4173-b98d-598dcc663532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.596471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5742f-929b-4173-b98d-598dcc663532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.597236 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5742f-929b-4173-b98d-598dcc663532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.598928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5742f-929b-4173-b98d-598dcc663532 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.608982 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1237/2500] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:57.613737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a354f6-21c1-42ef-8d33-247a64f36dfe admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.623667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a354f6-21c1-42ef-8d33-247a64f36dfe admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.628886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b7890-bee1-4711-be78-ab80ac827571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.629768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b7890-bee1-4711-be78-ab80ac827571 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.630047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b7890-bee1-4711-be78-ab80ac827571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.630319 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b7890-bee1-4711-be78-ab80ac827571 admin admin] PATH_INFO: `/v3/projects/7e4f98e1534a404f87ff2605c501adfd/users/47f93af9c51d4abca7eee374d20f0835/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.644580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b7890-bee1-4711-be78-ab80ac827571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4f98e1534a404f87ff2605c501adfd, user_id=47f93af9c51d4abca7eee374d20f0835, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.647242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b7890-bee1-4711-be78-ab80ac827571 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.660389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1264/2501] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:57 2022] PUT /identity/v3/projects/7e4f98e1534a404f87ff2605c501adfd/users/47f93af9c51d4abca7eee374d20f0835/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:57.671525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32760d9e-8d15-43bc-a494-47a8710f8436 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.671821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32760d9e-8d15-43bc-a494-47a8710f8436 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.672106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32760d9e-8d15-43bc-a494-47a8710f8436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.701710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-32760d9e-8d15-43bc-a494-47a8710f8436 None None] MFA Rules not processed for user `47f93af9c51d4abca7eee374d20f0835`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:57.702919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-32760d9e-8d15-43bc-a494-47a8710f8436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:57.766776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1238/2502] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:36:57.779167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b7890-bee1-4711-be78-ab80ac827571 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.780912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-c63b7890-bee1-4711-be78-ab80ac827571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:57.837381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b7890-bee1-4711-be78-ab80ac827571 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.840991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea66330-bc9e-408d-9694-077d71100fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.841621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea66330-bc9e-408d-9694-077d71100fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.841903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea66330-bc9e-408d-9694-077d71100fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.842174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea66330-bc9e-408d-9694-077d71100fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.850730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea66330-bc9e-408d-9694-077d71100fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.852147 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea66330-bc9e-408d-9694-077d71100fd4 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.861491 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1265/2503] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:58.295293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32760d9e-8d15-43bc-a494-47a8710f8436 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:58.295293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-32760d9e-8d15-43bc-a494-47a8710f8436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:58.369575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32760d9e-8d15-43bc-a494-47a8710f8436 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:58.374151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:58.375021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.375314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.375612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] PATH_INFO: `/v3/users/80870555a9a84101a7fade0e629c0b49` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.376970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80870555a9a84101a7fade0e629c0b49)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:58.379104 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:58.431606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80870555a9a84101a7fade0e629c0b49'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:58.432248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80870555a9a84101a7fade0e629c0b49'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:58.439564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80870555a9a84101a7fade0e629c0b49'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:58.440336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80870555a9a84101a7fade0e629c0b49'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:58.446045 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:58.456663 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80870555a9a84101a7fade0e629c0b49'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:58.478211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1239/2504] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:36:58 2022] DELETE /identity/v3/users/80870555a9a84101a7fade0e629c0b49 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:58.482791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a3697-ec36-458d-82e5-c3e8ba7c2b50 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.483089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a3697-ec36-458d-82e5-c3e8ba7c2b50 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.483372 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a3697-ec36-458d-82e5-c3e8ba7c2b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.490432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:58.509303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-943a3697-ec36-458d-82e5-c3e8ba7c2b50 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:58.510537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-943a3697-ec36-458d-82e5-c3e8ba7c2b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:58.586714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1266/2505] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:58.597248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:58.598733 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:58.656226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc9c27-7634-4ffa-b0d0-ab9440a111a5 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:58.660440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632d4ce-a7c4-480e-a2a2-03012ec7e1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:58.661135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632d4ce-a7c4-480e-a2a2-03012ec7e1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.661423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632d4ce-a7c4-480e-a2a2-03012ec7e1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.661700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632d4ce-a7c4-480e-a2a2-03012ec7e1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.670739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632d4ce-a7c4-480e-a2a2-03012ec7e1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:58.672189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632d4ce-a7c4-480e-a2a2-03012ec7e1c7 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:58.681685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1240/2506] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:36:58.954005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a3697-ec36-458d-82e5-c3e8ba7c2b50 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:58.956049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-943a3697-ec36-458d-82e5-c3e8ba7c2b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:58.968421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a3697-ec36-458d-82e5-c3e8ba7c2b50 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:58.973357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:58.974225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.974509 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.974783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] PATH_INFO: `/v3/projects/7b5d304bbc7742a5a27808981fd17ad2` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.975968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5d304bbc7742a5a27808981fd17ad2)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:58.977904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.024512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5d304bbc7742a5a27808981fd17ad2', 'request_id': 'req-3ce47e78-ef60-446d-af3b-b36f72a3eae5'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:59.031131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5d304bbc7742a5a27808981fd17ad2', 'request_id': 'req-3ce47e78-ef60-446d-af3b-b36f72a3eae5'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:59.037340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5d304bbc7742a5a27808981fd17ad2', 'request_id': 'req-3ce47e78-ef60-446d-af3b-b36f72a3eae5'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:59.050573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] The token cache is being invalidate because project 7b5d304bbc7742a5a27808981fd17ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:36:59.050900 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:59.052225 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-3ce47e78-ef60-446d-af3b-b36f72a3eae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:36:59.053815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1267/2507] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:36:58 2022] DELETE /identity/v3/projects/7b5d304bbc7742a5a27808981fd17ad2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:59.058706 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12f1a-70fb-419d-ae84-04bc6f3ad586 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.058999 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12f1a-70fb-419d-ae84-04bc6f3ad586 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.059316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12f1a-70fb-419d-ae84-04bc6f3ad586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.077653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-b4a12f1a-70fb-419d-ae84-04bc6f3ad586 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:59.078866 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b4a12f1a-70fb-419d-ae84-04bc6f3ad586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:59.138412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1241/2508] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:59.146055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55532f-7146-4d5f-a2e4-599dc8766997 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.146375 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55532f-7146-4d5f-a2e4-599dc8766997 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.146655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55532f-7146-4d5f-a2e4-599dc8766997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.169127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0b55532f-7146-4d5f-a2e4-599dc8766997 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:59.170538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0b55532f-7146-4d5f-a2e4-599dc8766997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:59.178959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1268/2509] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:59.185253 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a12f1a-70fb-419d-ae84-04bc6f3ad586 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.200626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a12f1a-70fb-419d-ae84-04bc6f3ad586 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.205895 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b0f55-2b18-4d4a-a410-b3208ce68d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.206585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b0f55-2b18-4d4a-a410-b3208ce68d28 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.207106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b0f55-2b18-4d4a-a410-b3208ce68d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.207395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b0f55-2b18-4d4a-a410-b3208ce68d28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.208241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b0f55-2b18-4d4a-a410-b3208ce68d28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:59.208436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b0f55-2b18-4d4a-a410-b3208ce68d28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.210089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b0f55-2b18-4d4a-a410-b3208ce68d28 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.222774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1242/2510] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:59.227096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55532f-7146-4d5f-a2e4-599dc8766997 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.237357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55532f-7146-4d5f-a2e4-599dc8766997 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.243603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bc2b-e5cb-49c3-9eab-9a8250ea300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.244271 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bc2b-e5cb-49c3-9eab-9a8250ea300d admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.244903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bc2b-e5cb-49c3-9eab-9a8250ea300d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.245448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bc2b-e5cb-49c3-9eab-9a8250ea300d admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.246080 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6bc2b-e5cb-49c3-9eab-9a8250ea300d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.247991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6bc2b-e5cb-49c3-9eab-9a8250ea300d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.283289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1269/2511] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:59 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:59.287242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b0f55-2b18-4d4a-a410-b3208ce68d28 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.297681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b0f55-2b18-4d4a-a410-b3208ce68d28 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.383390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185c5b4-e246-4104-8c4e-a0be04935521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.384546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185c5b4-e246-4104-8c4e-a0be04935521 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.385090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185c5b4-e246-4104-8c4e-a0be04935521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.385604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185c5b4-e246-4104-8c4e-a0be04935521 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.386924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185c5b4-e246-4104-8c4e-a0be04935521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.391109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185c5b4-e246-4104-8c4e-a0be04935521 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.413915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:59.420070 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1243/2512] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:59 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:59.424847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bc2b-e5cb-49c3-9eab-9a8250ea300d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.435537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bc2b-e5cb-49c3-9eab-9a8250ea300d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.441180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6eb05-77a5-4bf3-953f-58831e57aed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.441925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6eb05-77a5-4bf3-953f-58831e57aed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.442282 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6eb05-77a5-4bf3-953f-58831e57aed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.442635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6eb05-77a5-4bf3-953f-58831e57aed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.443441 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6eb05-77a5-4bf3-953f-58831e57aed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.445393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6eb05-77a5-4bf3-953f-58831e57aed8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.460596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1270/2513] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:59.462022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185c5b4-e246-4104-8c4e-a0be04935521 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.472807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185c5b4-e246-4104-8c4e-a0be04935521 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.477768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfa387-80fe-4f9f-b3cf-24b1a75a687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.478475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfa387-80fe-4f9f-b3cf-24b1a75a687d admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.478759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfa387-80fe-4f9f-b3cf-24b1a75a687d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.479037 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfa387-80fe-4f9f-b3cf-24b1a75a687d admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.479822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfa387-80fe-4f9f-b3cf-24b1a75a687d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.481467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfa387-80fe-4f9f-b3cf-24b1a75a687d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.496286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1244/2514] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:59.497556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6eb05-77a5-4bf3-953f-58831e57aed8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.507944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6eb05-77a5-4bf3-953f-58831e57aed8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.512732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61764ad-00a1-4ee9-93b5-8f33833a48e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.513652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61764ad-00a1-4ee9-93b5-8f33833a48e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.513940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61764ad-00a1-4ee9-93b5-8f33833a48e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.514223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61764ad-00a1-4ee9-93b5-8f33833a48e3 admin admin] PATH_INFO: `/v3/projects/9eb131e588084a328cfcbec2af85b84e/users/f45b0b4d1d1e417abd99b2f18e18d6f5/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.530520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61764ad-00a1-4ee9-93b5-8f33833a48e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb131e588084a328cfcbec2af85b84e, user_id=f45b0b4d1d1e417abd99b2f18e18d6f5, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.533751 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61764ad-00a1-4ee9-93b5-8f33833a48e3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.548588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1271/2515] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:59 2022] PUT /identity/v3/projects/9eb131e588084a328cfcbec2af85b84e/users/f45b0b4d1d1e417abd99b2f18e18d6f5/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:59.559745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4ddf7-93f2-49a1-bee7-5cfec27e488b None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.560041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4ddf7-93f2-49a1-bee7-5cfec27e488b None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.560323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4ddf7-93f2-49a1-bee7-5cfec27e488b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.589955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-f3b4ddf7-93f2-49a1-bee7-5cfec27e488b None None] MFA Rules not processed for user `f45b0b4d1d1e417abd99b2f18e18d6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:59.593807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f3b4ddf7-93f2-49a1-bee7-5cfec27e488b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:59.676352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1245/2516] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:36:59.680882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61764ad-00a1-4ee9-93b5-8f33833a48e3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.707461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:59.747262 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61764ad-00a1-4ee9-93b5-8f33833a48e3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.752616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ce8c2-518a-4d5b-8dfa-b77cbfc2acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.753328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ce8c2-518a-4d5b-8dfa-b77cbfc2acb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.753619 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ce8c2-518a-4d5b-8dfa-b77cbfc2acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.753897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ce8c2-518a-4d5b-8dfa-b77cbfc2acb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.754658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ce8c2-518a-4d5b-8dfa-b77cbfc2acb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.756563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ce8c2-518a-4d5b-8dfa-b77cbfc2acb4 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.786162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1272/2517] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:36:59 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:59.790740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4ddf7-93f2-49a1-bee7-5cfec27e488b None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.803158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4ddf7-93f2-49a1-bee7-5cfec27e488b None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.866191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bddf8-4588-40c5-95e8-48a2c6d0f5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.866873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bddf8-4588-40c5-95e8-48a2c6d0f5e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.867158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bddf8-4588-40c5-95e8-48a2c6d0f5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.867547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bddf8-4588-40c5-95e8-48a2c6d0f5e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.868115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bddf8-4588-40c5-95e8-48a2c6d0f5e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.870096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bddf8-4588-40c5-95e8-48a2c6d0f5e9 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.890182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1246/2518] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:36:59 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:59.894892 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ce8c2-518a-4d5b-8dfa-b77cbfc2acb4 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.905788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ce8c2-518a-4d5b-8dfa-b77cbfc2acb4 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.911367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66f0e0-fa76-4c42-9dc6-485ac09390f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.912064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66f0e0-fa76-4c42-9dc6-485ac09390f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.912347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66f0e0-fa76-4c42-9dc6-485ac09390f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.912624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66f0e0-fa76-4c42-9dc6-485ac09390f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.913391 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66f0e0-fa76-4c42-9dc6-485ac09390f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.914983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66f0e0-fa76-4c42-9dc6-485ac09390f3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.926277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1273/2519] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:36:59.930535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bddf8-4588-40c5-95e8-48a2c6d0f5e9 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.940992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bddf8-4588-40c5-95e8-48a2c6d0f5e9 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.945612 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03819b30-943f-46b4-a1d5-0eefdca35d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.946647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03819b30-943f-46b4-a1d5-0eefdca35d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.946955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03819b30-943f-46b4-a1d5-0eefdca35d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.947227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03819b30-943f-46b4-a1d5-0eefdca35d2c admin admin] PATH_INFO: `/v3/projects/0a0675fefaad4b039b9cdd22c598c972/users/de060964367d402ebd7cf4712c2fefbc/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.965099 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03819b30-943f-46b4-a1d5-0eefdca35d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0675fefaad4b039b9cdd22c598c972, user_id=de060964367d402ebd7cf4712c2fefbc, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.968040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03819b30-943f-46b4-a1d5-0eefdca35d2c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.993812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1247/2520] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:36:59 2022] PUT /identity/v3/projects/0a0675fefaad4b039b9cdd22c598c972/users/de060964367d402ebd7cf4712c2fefbc/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:59.998183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66f0e0-fa76-4c42-9dc6-485ac09390f3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.008522 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66f0e0-fa76-4c42-9dc6-485ac09390f3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.071643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b25fda-8b52-475f-8631-e2c3b56b305b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.072311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25fda-8b52-475f-8631-e2c3b56b305b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.072600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25fda-8b52-475f-8631-e2c3b56b305b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.072896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25fda-8b52-475f-8631-e2c3b56b305b admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.073644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b25fda-8b52-475f-8631-e2c3b56b305b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:00.075333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b25fda-8b52-475f-8631-e2c3b56b305b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:00.088918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1274/2521] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:00.091233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03819b30-943f-46b4-a1d5-0eefdca35d2c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.101996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03819b30-943f-46b4-a1d5-0eefdca35d2c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.106803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266d203-20ce-475b-8a94-86a2024f2052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.107788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266d203-20ce-475b-8a94-86a2024f2052 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.107788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266d203-20ce-475b-8a94-86a2024f2052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.108074 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266d203-20ce-475b-8a94-86a2024f2052 admin admin] PATH_INFO: `/v3/projects/0a0675fefaad4b039b9cdd22c598c972/users/de060964367d402ebd7cf4712c2fefbc/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.110076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266d203-20ce-475b-8a94-86a2024f2052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0675fefaad4b039b9cdd22c598c972, user_id=de060964367d402ebd7cf4712c2fefbc, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:00.112758 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266d203-20ce-475b-8a94-86a2024f2052 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:00.123628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1248/2522] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:00 2022] PUT /identity/v3/projects/0a0675fefaad4b039b9cdd22c598c972/users/de060964367d402ebd7cf4712c2fefbc/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:00.135037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2eb734-02e7-4c76-80d9-2c07c9b75d4a None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.135331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2eb734-02e7-4c76-80d9-2c07c9b75d4a None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.135667 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2eb734-02e7-4c76-80d9-2c07c9b75d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.163719 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-7b2eb734-02e7-4c76-80d9-2c07c9b75d4a None None] MFA Rules not processed for user `de060964367d402ebd7cf4712c2fefbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:00.164848 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7b2eb734-02e7-4c76-80d9-2c07c9b75d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:00.233028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1275/2523] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:00.244366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266d203-20ce-475b-8a94-86a2024f2052 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.246100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-2266d203-20ce-475b-8a94-86a2024f2052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:00.280711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:00.305530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266d203-20ce-475b-8a94-86a2024f2052 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.309285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f5e5d-6aeb-4183-8add-955be5790aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.309978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f5e5d-6aeb-4183-8add-955be5790aab service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.310262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f5e5d-6aeb-4183-8add-955be5790aab service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.310536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f5e5d-6aeb-4183-8add-955be5790aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.320278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f5e5d-6aeb-4183-8add-955be5790aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:00.321761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f5e5d-6aeb-4183-8add-955be5790aab service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:00.331271 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1249/2524] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:00.412834 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2eb734-02e7-4c76-80d9-2c07c9b75d4a None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.423380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2eb734-02e7-4c76-80d9-2c07c9b75d4a None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.428825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d2cb4-cb42-4b55-bb75-668c9c6e9b09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.429537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d2cb4-cb42-4b55-bb75-668c9c6e9b09 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.429642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f5e5d-6aeb-4183-8add-955be5790aab service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.429877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d2cb4-cb42-4b55-bb75-668c9c6e9b09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.430164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d2cb4-cb42-4b55-bb75-668c9c6e9b09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.431373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3c1f5e5d-6aeb-4183-8add-955be5790aab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:00.446041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:00.495569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d2cb4-cb42-4b55-bb75-668c9c6e9b09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:00.497127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d2cb4-cb42-4b55-bb75-668c9c6e9b09 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:00.501538 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f5e5d-6aeb-4183-8add-955be5790aab service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.506399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1276/2525] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:00.508124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.509035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.509327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.509605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] PATH_INFO: `/v3/users/47f93af9c51d4abca7eee374d20f0835` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.510826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f93af9c51d4abca7eee374d20f0835)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:00.512829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:00.571242 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f93af9c51d4abca7eee374d20f0835'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:00.572289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f93af9c51d4abca7eee374d20f0835'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:00.581172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f93af9c51d4abca7eee374d20f0835'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:00.581921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f93af9c51d4abca7eee374d20f0835'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:00.598832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f93af9c51d4abca7eee374d20f0835'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:00.618156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1250/2526] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:00 2022] DELETE /identity/v3/users/47f93af9c51d4abca7eee374d20f0835 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:00.622055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b8cad-dbd0-4c50-9432-bd3ed45b3a80 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.622337 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b8cad-dbd0-4c50-9432-bd3ed45b3a80 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.622680 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b8cad-dbd0-4c50-9432-bd3ed45b3a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.642187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6f9b8cad-dbd0-4c50-9432-bd3ed45b3a80 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:00.643892 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6f9b8cad-dbd0-4c50-9432-bd3ed45b3a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:00.712902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1277/2527] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:00.723696 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.725443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:00.786691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c101cd4-31a2-435f-a9d6-fc75f9633380 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.790342 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa119e7a-9224-46f3-90f4-265d5a598819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.790990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa119e7a-9224-46f3-90f4-265d5a598819 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.791273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa119e7a-9224-46f3-90f4-265d5a598819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.791547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa119e7a-9224-46f3-90f4-265d5a598819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.799006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:00.801647 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa119e7a-9224-46f3-90f4-265d5a598819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:00.804654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa119e7a-9224-46f3-90f4-265d5a598819 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:00.819292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1251/2528] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:01.086771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b8cad-dbd0-4c50-9432-bd3ed45b3a80 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.089030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6f9b8cad-dbd0-4c50-9432-bd3ed45b3a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:01.105998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b8cad-dbd0-4c50-9432-bd3ed45b3a80 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.112374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.112902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.113205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.113483 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] PATH_INFO: `/v3/projects/7e4f98e1534a404f87ff2605c501adfd` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.114715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4f98e1534a404f87ff2605c501adfd)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.116672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.170270 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4f98e1534a404f87ff2605c501adfd', 'request_id': 'req-a453db89-b251-4327-85de-1b5ba4046631'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:01.177754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4f98e1534a404f87ff2605c501adfd', 'request_id': 'req-a453db89-b251-4327-85de-1b5ba4046631'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:01.184514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4f98e1534a404f87ff2605c501adfd', 'request_id': 'req-a453db89-b251-4327-85de-1b5ba4046631'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:01.204108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] The token cache is being invalidate because project 7e4f98e1534a404f87ff2605c501adfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:01.204283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:01.205111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-a453db89-b251-4327-85de-1b5ba4046631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:01.207907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1278/2529] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:01 2022] DELETE /identity/v3/projects/7e4f98e1534a404f87ff2605c501adfd => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:01.212903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72ae5-d0e0-4f7a-b69d-b5a04895f9e8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.213339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72ae5-d0e0-4f7a-b69d-b5a04895f9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.213458 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72ae5-d0e0-4f7a-b69d-b5a04895f9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.232898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-fde72ae5-d0e0-4f7a-b69d-b5a04895f9e8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:01.234178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-fde72ae5-d0e0-4f7a-b69d-b5a04895f9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:01.309090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1252/2530] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:01.320452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f00291-82a3-4324-88f2-919b299cde5c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.320758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f00291-82a3-4324-88f2-919b299cde5c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.321046 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f00291-82a3-4324-88f2-919b299cde5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.344209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-43f00291-82a3-4324-88f2-919b299cde5c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:01.345657 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-43f00291-82a3-4324-88f2-919b299cde5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:01.354201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1279/2531] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:01.358592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde72ae5-d0e0-4f7a-b69d-b5a04895f9e8 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.369311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde72ae5-d0e0-4f7a-b69d-b5a04895f9e8 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.374320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5590ce-07e7-4304-a2a5-aee6c78f5d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.376267 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5590ce-07e7-4304-a2a5-aee6c78f5d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.376620 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5590ce-07e7-4304-a2a5-aee6c78f5d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.377551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5590ce-07e7-4304-a2a5-aee6c78f5d29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.378375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5590ce-07e7-4304-a2a5-aee6c78f5d29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:01.378755 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5590ce-07e7-4304-a2a5-aee6c78f5d29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.380465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5590ce-07e7-4304-a2a5-aee6c78f5d29 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.396931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1253/2532] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:01.398160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f00291-82a3-4324-88f2-919b299cde5c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.413915 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f00291-82a3-4324-88f2-919b299cde5c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.421130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6af04d-5c9a-4fb1-8156-f9785665b631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.421955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6af04d-5c9a-4fb1-8156-f9785665b631 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.422517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6af04d-5c9a-4fb1-8156-f9785665b631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.422888 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6af04d-5c9a-4fb1-8156-f9785665b631 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.424851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6af04d-5c9a-4fb1-8156-f9785665b631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.427817 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6af04d-5c9a-4fb1-8156-f9785665b631 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.461050 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1280/2533] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:01 2022] POST /identity/v3/projects => generated 399 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:01.464949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5590ce-07e7-4304-a2a5-aee6c78f5d29 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.476303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5590ce-07e7-4304-a2a5-aee6c78f5d29 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.540783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fbcd6-65f3-435a-b240-24647e47d781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.541531 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fbcd6-65f3-435a-b240-24647e47d781 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.541813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fbcd6-65f3-435a-b240-24647e47d781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.542090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fbcd6-65f3-435a-b240-24647e47d781 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.542788 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fbcd6-65f3-435a-b240-24647e47d781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.544759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fbcd6-65f3-435a-b240-24647e47d781 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.573874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1254/2534] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:01 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:01.575177 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6af04d-5c9a-4fb1-8156-f9785665b631 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.586917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6af04d-5c9a-4fb1-8156-f9785665b631 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.591925 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e6fd3-1268-4d4f-84a6-daf9d12048d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.592791 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e6fd3-1268-4d4f-84a6-daf9d12048d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.593201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e6fd3-1268-4d4f-84a6-daf9d12048d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.593615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e6fd3-1268-4d4f-84a6-daf9d12048d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.594423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e6fd3-1268-4d4f-84a6-daf9d12048d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.597092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e6fd3-1268-4d4f-84a6-daf9d12048d2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.610410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1281/2535] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:01.610672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:01.614405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fbcd6-65f3-435a-b240-24647e47d781 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.626383 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fbcd6-65f3-435a-b240-24647e47d781 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.631541 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12ad47-e737-470c-8b79-ae62130b3d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.632181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12ad47-e737-470c-8b79-ae62130b3d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.632468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12ad47-e737-470c-8b79-ae62130b3d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.632744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12ad47-e737-470c-8b79-ae62130b3d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.633522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12ad47-e737-470c-8b79-ae62130b3d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.635201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12ad47-e737-470c-8b79-ae62130b3d09 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.647421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1255/2536] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:01.651696 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e6fd3-1268-4d4f-84a6-daf9d12048d2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.663016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e6fd3-1268-4d4f-84a6-daf9d12048d2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.667965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27eb94-30d4-4d93-b3cb-16fb19fcac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.668904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27eb94-30d4-4d93-b3cb-16fb19fcac98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.669220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27eb94-30d4-4d93-b3cb-16fb19fcac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.669470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27eb94-30d4-4d93-b3cb-16fb19fcac98 admin admin] PATH_INFO: `/v3/projects/84574144502943a988b912b1ac29b3db/users/a5590ee3563b4356bc87bc1824b00dbf/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.693397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27eb94-30d4-4d93-b3cb-16fb19fcac98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84574144502943a988b912b1ac29b3db, user_id=a5590ee3563b4356bc87bc1824b00dbf, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.696059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27eb94-30d4-4d93-b3cb-16fb19fcac98 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.721557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1282/2537] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:01 2022] PUT /identity/v3/projects/84574144502943a988b912b1ac29b3db/users/a5590ee3563b4356bc87bc1824b00dbf/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:01.779215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acd193-0ab2-4671-82f9-1e66c66d3c75 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.779736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acd193-0ab2-4671-82f9-1e66c66d3c75 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.779831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acd193-0ab2-4671-82f9-1e66c66d3c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.819742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-28acd193-0ab2-4671-82f9-1e66c66d3c75 None None] MFA Rules not processed for user `a5590ee3563b4356bc87bc1824b00dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:01.821905 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-28acd193-0ab2-4671-82f9-1e66c66d3c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:01.883606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:01.924915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1256/2538] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:01.936230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27eb94-30d4-4d93-b3cb-16fb19fcac98 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.939765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2f27eb94-30d4-4d93-b3cb-16fb19fcac98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:02.054328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27eb94-30d4-4d93-b3cb-16fb19fcac98 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:02.069423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349ad77-e692-4aba-905e-8b535c200512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:02.070771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349ad77-e692-4aba-905e-8b535c200512 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:02.071153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349ad77-e692-4aba-905e-8b535c200512 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:02.071659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349ad77-e692-4aba-905e-8b535c200512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:02.085013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349ad77-e692-4aba-905e-8b535c200512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:02.086828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349ad77-e692-4aba-905e-8b535c200512 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:02.134114 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1283/2539] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:11.339949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acd193-0ab2-4671-82f9-1e66c66d3c75 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:11.341381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-28acd193-0ab2-4671-82f9-1e66c66d3c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:11.401384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acd193-0ab2-4671-82f9-1e66c66d3c75 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:11.405022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21122193-92bf-45cf-a35d-89219670cbff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:11.405661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21122193-92bf-45cf-a35d-89219670cbff service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:11.405943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21122193-92bf-45cf-a35d-89219670cbff service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:11.406217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21122193-92bf-45cf-a35d-89219670cbff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:11.407967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-21122193-92bf-45cf-a35d-89219670cbff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:11.477746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21122193-92bf-45cf-a35d-89219670cbff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:11.479224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21122193-92bf-45cf-a35d-89219670cbff service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:11.489112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1257/2540] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:12.097422 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:12.409679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349ad77-e692-4aba-905e-8b535c200512 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:12.422287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349ad77-e692-4aba-905e-8b535c200512 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:12.427843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:12.429045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:12.429537 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:12.429890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] PATH_INFO: `/v3/users/a5590ee3563b4356bc87bc1824b00dbf` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:12.431212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5590ee3563b4356bc87bc1824b00dbf)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:12.433284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:12.490636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5590ee3563b4356bc87bc1824b00dbf'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:12.509962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5590ee3563b4356bc87bc1824b00dbf'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:12.518955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5590ee3563b4356bc87bc1824b00dbf'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:12.519681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5590ee3563b4356bc87bc1824b00dbf'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:12.520289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5590ee3563b4356bc87bc1824b00dbf'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:12.535563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1284/2541] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:12 2022] DELETE /identity/v3/users/a5590ee3563b4356bc87bc1824b00dbf => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:12.541172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9c4c4-00a4-46ad-87e6-99f7b05ff72c None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:12.541719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9c4c4-00a4-46ad-87e6-99f7b05ff72c None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:12.541814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9c4c4-00a4-46ad-87e6-99f7b05ff72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:12.562535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-02e9c4c4-00a4-46ad-87e6-99f7b05ff72c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:12.563938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-02e9c4c4-00a4-46ad-87e6-99f7b05ff72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:12.639393 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1258/2542] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:12.650519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:12.652184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:12.714190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9ff05-4db3-45d1-afd8-2a4dd1d9a3b1 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:12.717892 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6a9bc-0026-442c-bc57-204827e300c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:12.718554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6a9bc-0026-442c-bc57-204827e300c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:12.718839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6a9bc-0026-442c-bc57-204827e300c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:12.719141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6a9bc-0026-442c-bc57-204827e300c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:12.729179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6a9bc-0026-442c-bc57-204827e300c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:12.730637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6a9bc-0026-442c-bc57-204827e300c3 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:12.741755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1285/2543] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:13.042057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9c4c4-00a4-46ad-87e6-99f7b05ff72c None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:13.043654 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-02e9c4c4-00a4-46ad-87e6-99f7b05ff72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:13.065599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9c4c4-00a4-46ad-87e6-99f7b05ff72c None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:13.072680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:13.073978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:13.074476 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:13.074974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] PATH_INFO: `/v3/projects/84574144502943a988b912b1ac29b3db` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:13.076427 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84574144502943a988b912b1ac29b3db)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:13.078421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:13.144661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:13.166196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84574144502943a988b912b1ac29b3db', 'request_id': 'req-d03a9f3c-6158-4fb7-9bc2-5370027652e1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:13.175871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84574144502943a988b912b1ac29b3db', 'request_id': 'req-d03a9f3c-6158-4fb7-9bc2-5370027652e1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:13.185371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84574144502943a988b912b1ac29b3db', 'request_id': 'req-d03a9f3c-6158-4fb7-9bc2-5370027652e1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:13.201807 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] The token cache is being invalidate because project 84574144502943a988b912b1ac29b3db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:13.202314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:13.203756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:13.205953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1259/2544] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:13 2022] DELETE /identity/v3/projects/84574144502943a988b912b1ac29b3db => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:15.091736 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6a9bc-0026-442c-bc57-204827e300c3 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.093299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-35e6a9bc-0026-442c-bc57-204827e300c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:15.112664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6a9bc-0026-442c-bc57-204827e300c3 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.178359 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.179170 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.179450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.179723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] PATH_INFO: `/v3/users/139e3e42acff4c4ca8df0a7c4fc81053` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.180935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=139e3e42acff4c4ca8df0a7c4fc81053)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.182835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.200413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:15.233716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '139e3e42acff4c4ca8df0a7c4fc81053'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.246740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '139e3e42acff4c4ca8df0a7c4fc81053'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.256008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '139e3e42acff4c4ca8df0a7c4fc81053'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.256715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '139e3e42acff4c4ca8df0a7c4fc81053'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.257414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-7f31b702-e232-4484-b99b-f65be77742c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '139e3e42acff4c4ca8df0a7c4fc81053'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.271475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1286/2545] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:15 2022] DELETE /identity/v3/users/139e3e42acff4c4ca8df0a7c4fc81053 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:15.448565 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.455144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:15.488354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a9f3c-6158-4fb7-9bc2-5370027652e1 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.548220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.549313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.549621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.550071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] PATH_INFO: `/v3/projects/65c6b1563c424330ade1ef15f8efa110` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.551425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c6b1563c424330ade1ef15f8efa110)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.553395 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.602390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c6b1563c424330ade1ef15f8efa110', 'request_id': 'req-fde7c00d-e054-48b0-86ed-87329b374559'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.608632 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c6b1563c424330ade1ef15f8efa110', 'request_id': 'req-fde7c00d-e054-48b0-86ed-87329b374559'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.616991 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c6b1563c424330ade1ef15f8efa110', 'request_id': 'req-fde7c00d-e054-48b0-86ed-87329b374559'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.628810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] The token cache is being invalidate because project 65c6b1563c424330ade1ef15f8efa110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:15.629132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.629763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-fde7c00d-e054-48b0-86ed-87329b374559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:15.631301 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1260/2546] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:15 2022] DELETE /identity/v3/projects/65c6b1563c424330ade1ef15f8efa110 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:15.636525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dce824-d481-4e6a-b223-2f127fd4405b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.636821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dce824-d481-4e6a-b223-2f127fd4405b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.637146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dce824-d481-4e6a-b223-2f127fd4405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.656424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-17dce824-d481-4e6a-b223-2f127fd4405b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:15.657789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-17dce824-d481-4e6a-b223-2f127fd4405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:15.720689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1287/2547] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:15.728759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56dc9-f69b-43d2-947b-099508c2b322 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.729083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56dc9-f69b-43d2-947b-099508c2b322 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.729361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56dc9-f69b-43d2-947b-099508c2b322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.748133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a6b56dc9-f69b-43d2-947b-099508c2b322 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:15.749371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a6b56dc9-f69b-43d2-947b-099508c2b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:15.757977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1261/2548] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:15.763190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dce824-d481-4e6a-b223-2f127fd4405b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.774332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dce824-d481-4e6a-b223-2f127fd4405b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.779200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f5715-f6f7-48ac-8111-a94c2e64414a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.779854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f5715-f6f7-48ac-8111-a94c2e64414a admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.780139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f5715-f6f7-48ac-8111-a94c2e64414a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.780414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f5715-f6f7-48ac-8111-a94c2e64414a admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.781181 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f5715-f6f7-48ac-8111-a94c2e64414a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:15.781487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f5715-f6f7-48ac-8111-a94c2e64414a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.783082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f5715-f6f7-48ac-8111-a94c2e64414a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.798219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1288/2549] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:15.800455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b56dc9-f69b-43d2-947b-099508c2b322 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.810078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b56dc9-f69b-43d2-947b-099508c2b322 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.815970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121418e2-6d0d-4f8e-8a31-149e9a4918c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.816625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121418e2-6d0d-4f8e-8a31-149e9a4918c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.816986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121418e2-6d0d-4f8e-8a31-149e9a4918c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.817223 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121418e2-6d0d-4f8e-8a31-149e9a4918c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.817927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121418e2-6d0d-4f8e-8a31-149e9a4918c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.819871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121418e2-6d0d-4f8e-8a31-149e9a4918c5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.849196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1262/2550] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:15 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:15.850603 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f5715-f6f7-48ac-8111-a94c2e64414a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.861048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f5715-f6f7-48ac-8111-a94c2e64414a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.922437 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adec361-657a-4e05-9f33-478a92c2226d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.923720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adec361-657a-4e05-9f33-478a92c2226d admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.923720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adec361-657a-4e05-9f33-478a92c2226d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.924774 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adec361-657a-4e05-9f33-478a92c2226d admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.925595 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adec361-657a-4e05-9f33-478a92c2226d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.928099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adec361-657a-4e05-9f33-478a92c2226d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.955908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1289/2551] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:15 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:15.960155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121418e2-6d0d-4f8e-8a31-149e9a4918c5 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.972272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121418e2-6d0d-4f8e-8a31-149e9a4918c5 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.976952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e4616-5b75-4655-ae29-5e0944fe4828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.977618 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e4616-5b75-4655-ae29-5e0944fe4828 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.977903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e4616-5b75-4655-ae29-5e0944fe4828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.978181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e4616-5b75-4655-ae29-5e0944fe4828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.978946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e4616-5b75-4655-ae29-5e0944fe4828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.980556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e4616-5b75-4655-ae29-5e0944fe4828 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.992170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1263/2552] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:15.996161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adec361-657a-4e05-9f33-478a92c2226d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.006208 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adec361-657a-4e05-9f33-478a92c2226d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.010872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618764d1-5e1c-4641-be4a-bfdd9cef5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.011532 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618764d1-5e1c-4641-be4a-bfdd9cef5e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.011818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618764d1-5e1c-4641-be4a-bfdd9cef5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.012089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618764d1-5e1c-4641-be4a-bfdd9cef5e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.012816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618764d1-5e1c-4641-be4a-bfdd9cef5e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.015344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618764d1-5e1c-4641-be4a-bfdd9cef5e9d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.020648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.029371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1290/2553] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:16.030568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e4616-5b75-4655-ae29-5e0944fe4828 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.042128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e4616-5b75-4655-ae29-5e0944fe4828 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.046801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164c4f0-ed50-4783-9d7d-8a6d3482bbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.047675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164c4f0-ed50-4783-9d7d-8a6d3482bbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.047952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164c4f0-ed50-4783-9d7d-8a6d3482bbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.048479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164c4f0-ed50-4783-9d7d-8a6d3482bbe7 admin admin] PATH_INFO: `/v3/projects/86083144dbcd42c491a8086e3800a6cd/users/eeb468d8929e4a3696bc180a565a030b/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.061117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.064411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164c4f0-ed50-4783-9d7d-8a6d3482bbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86083144dbcd42c491a8086e3800a6cd, user_id=eeb468d8929e4a3696bc180a565a030b, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.068278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164c4f0-ed50-4783-9d7d-8a6d3482bbe7 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.084353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1264/2554] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:16 2022] PUT /identity/v3/projects/86083144dbcd42c491a8086e3800a6cd/users/eeb468d8929e4a3696bc180a565a030b/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:16.095231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b12462-a69e-4477-b44e-59ca3eb51031 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.096305 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b12462-a69e-4477-b44e-59ca3eb51031 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.096408 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b12462-a69e-4477-b44e-59ca3eb51031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.125636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-85b12462-a69e-4477-b44e-59ca3eb51031 None None] MFA Rules not processed for user `eeb468d8929e4a3696bc180a565a030b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:16.127018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-85b12462-a69e-4477-b44e-59ca3eb51031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:16.201054 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1291/2555] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:16.212526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164c4f0-ed50-4783-9d7d-8a6d3482bbe7 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.214361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6164c4f0-ed50-4783-9d7d-8a6d3482bbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:16.314741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164c4f0-ed50-4783-9d7d-8a6d3482bbe7 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.320477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fa516-4f13-4578-b532-24add892e3c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.321161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fa516-4f13-4578-b532-24add892e3c5 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.321441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fa516-4f13-4578-b532-24add892e3c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.321716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fa516-4f13-4578-b532-24add892e3c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.329873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fa516-4f13-4578-b532-24add892e3c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.332103 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fa516-4f13-4578-b532-24add892e3c5 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.342785 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1265/2556] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:16.600732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b12462-a69e-4477-b44e-59ca3eb51031 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.602168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-85b12462-a69e-4477-b44e-59ca3eb51031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:16.688380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b12462-a69e-4477-b44e-59ca3eb51031 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.693421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.694299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.694590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.694865 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] PATH_INFO: `/v3/users/f35664e135bb4c338a46332f9e29343d` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.697144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f35664e135bb4c338a46332f9e29343d)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.699057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.760671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f35664e135bb4c338a46332f9e29343d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:16.773650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f35664e135bb4c338a46332f9e29343d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:16.781388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f35664e135bb4c338a46332f9e29343d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:16.782004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f35664e135bb4c338a46332f9e29343d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:16.782612 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f35664e135bb4c338a46332f9e29343d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:16.795356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1292/2557] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:16 2022] DELETE /identity/v3/users/f35664e135bb4c338a46332f9e29343d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:16.800087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390fd98-86ae-4504-bcf6-2651c8cea699 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.800581 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390fd98-86ae-4504-bcf6-2651c8cea699 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.800679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390fd98-86ae-4504-bcf6-2651c8cea699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.817829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-6390fd98-86ae-4504-bcf6-2651c8cea699 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:16.819016 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6390fd98-86ae-4504-bcf6-2651c8cea699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:16.875816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1266/2558] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:16.886382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.887993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:16.900077 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.952670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2ed16-40bb-4ea9-8d54-877ceeab3130 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.956970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc00b4-39c4-4821-ac95-9fc0abe08ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.957617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc00b4-39c4-4821-ac95-9fc0abe08ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.957903 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc00b4-39c4-4821-ac95-9fc0abe08ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.958182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc00b4-39c4-4821-ac95-9fc0abe08ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.967089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc00b4-39c4-4821-ac95-9fc0abe08ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.968518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc00b4-39c4-4821-ac95-9fc0abe08ec1 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.978688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1293/2559] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:17.251432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390fd98-86ae-4504-bcf6-2651c8cea699 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.253843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6390fd98-86ae-4504-bcf6-2651c8cea699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:17.266993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390fd98-86ae-4504-bcf6-2651c8cea699 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.273007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.273760 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.274046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.274324 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] PATH_INFO: `/v3/projects/ecb3a604bce84d20aa08c32faa4b4013` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.275527 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb3a604bce84d20aa08c32faa4b4013)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.277752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.326344 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb3a604bce84d20aa08c32faa4b4013', 'request_id': 'req-de9e8664-838f-42e6-89f1-5886c4fd2a48'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.332837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb3a604bce84d20aa08c32faa4b4013', 'request_id': 'req-de9e8664-838f-42e6-89f1-5886c4fd2a48'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.340616 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb3a604bce84d20aa08c32faa4b4013', 'request_id': 'req-de9e8664-838f-42e6-89f1-5886c4fd2a48'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.349323 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:17.360759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] The token cache is being invalidate because project ecb3a604bce84d20aa08c32faa4b4013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:17.361081 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.362246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.363923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1267/2560] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:17 2022] DELETE /identity/v3/projects/ecb3a604bce84d20aa08c32faa4b4013 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:17.367894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc00b4-39c4-4821-ac95-9fc0abe08ec1 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.379609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc00b4-39c4-4821-ac95-9fc0abe08ec1 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.439055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.440064 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.440486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.440761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] PATH_INFO: `/v3/users/fb31f4f301d448f8a2d0974306166fc8` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.441991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb31f4f301d448f8a2d0974306166fc8)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.443890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.492730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb31f4f301d448f8a2d0974306166fc8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.506410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb31f4f301d448f8a2d0974306166fc8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.513973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb31f4f301d448f8a2d0974306166fc8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.514754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb31f4f301d448f8a2d0974306166fc8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.515462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b69d6542-f603-4420-9fc1-d2cdbd8a9a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb31f4f301d448f8a2d0974306166fc8'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:17.529651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1294/2561] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:17 2022] DELETE /identity/v3/users/fb31f4f301d448f8a2d0974306166fc8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:17.843181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.845133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:17.861971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e8664-838f-42e6-89f1-5886c4fd2a48 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.952044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.952951 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.953238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.953516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] PATH_INFO: `/v3/projects/b9829182b3e44100b8bac79a58a650d5` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.954736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9829182b3e44100b8bac79a58a650d5)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.957479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.016535 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9829182b3e44100b8bac79a58a650d5', 'request_id': 'req-b15c8d61-455c-4d5f-8e71-9dcc3464e808'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:18.024433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9829182b3e44100b8bac79a58a650d5', 'request_id': 'req-b15c8d61-455c-4d5f-8e71-9dcc3464e808'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:18.035135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9829182b3e44100b8bac79a58a650d5', 'request_id': 'req-b15c8d61-455c-4d5f-8e71-9dcc3464e808'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:18.047311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] The token cache is being invalidate because project b9829182b3e44100b8bac79a58a650d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:18.047600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:18.048267 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b15c8d61-455c-4d5f-8e71-9dcc3464e808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:18.049974 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1268/2562] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:17 2022] DELETE /identity/v3/projects/b9829182b3e44100b8bac79a58a650d5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:18.054855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff54aa-86e0-4357-a130-9e4af07ee389 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.055146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff54aa-86e0-4357-a130-9e4af07ee389 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.055455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff54aa-86e0-4357-a130-9e4af07ee389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.074400 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-baff54aa-86e0-4357-a130-9e4af07ee389 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:18.075715 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-baff54aa-86e0-4357-a130-9e4af07ee389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:18.105448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:18.143682 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1295/2563] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:18.152257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34943ef0-5fd1-45a0-9778-c13d6d1d6e62 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.152551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34943ef0-5fd1-45a0-9778-c13d6d1d6e62 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.152823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34943ef0-5fd1-45a0-9778-c13d6d1d6e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.172457 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-34943ef0-5fd1-45a0-9778-c13d6d1d6e62 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:18.173669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-34943ef0-5fd1-45a0-9778-c13d6d1d6e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:18.182022 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1269/2564] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:18.186246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff54aa-86e0-4357-a130-9e4af07ee389 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.195879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff54aa-86e0-4357-a130-9e4af07ee389 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.200444 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11094d32-6a7b-4297-97aa-849e7a9d5754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.201128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11094d32-6a7b-4297-97aa-849e7a9d5754 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.201413 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11094d32-6a7b-4297-97aa-849e7a9d5754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.201688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11094d32-6a7b-4297-97aa-849e7a9d5754 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.202407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11094d32-6a7b-4297-97aa-849e7a9d5754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:18.202714 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11094d32-6a7b-4297-97aa-849e7a9d5754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.204296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11094d32-6a7b-4297-97aa-849e7a9d5754 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.216346 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1296/2565] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:18.220523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34943ef0-5fd1-45a0-9778-c13d6d1d6e62 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.230152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34943ef0-5fd1-45a0-9778-c13d6d1d6e62 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.234777 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a553413-0ac9-416b-a993-6347d8d19fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.235410 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a553413-0ac9-416b-a993-6347d8d19fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.235685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a553413-0ac9-416b-a993-6347d8d19fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.235954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a553413-0ac9-416b-a993-6347d8d19fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.236671 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a553413-0ac9-416b-a993-6347d8d19fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.238692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a553413-0ac9-416b-a993-6347d8d19fd6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.263772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1270/2566] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:18 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:18.268175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11094d32-6a7b-4297-97aa-849e7a9d5754 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.278745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11094d32-6a7b-4297-97aa-849e7a9d5754 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.343634 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdacc2-caa1-4841-bbf0-45a39d03f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.344621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdacc2-caa1-4841-bbf0-45a39d03f1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.345096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdacc2-caa1-4841-bbf0-45a39d03f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.345514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdacc2-caa1-4841-bbf0-45a39d03f1fe admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.347131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdacc2-caa1-4841-bbf0-45a39d03f1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.350256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdacc2-caa1-4841-bbf0-45a39d03f1fe admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.375812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1297/2567] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:18 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:18.379953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a553413-0ac9-416b-a993-6347d8d19fd6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.390309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a553413-0ac9-416b-a993-6347d8d19fd6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.395035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152707a-ccab-43e7-9287-3c433fbe6d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.395687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152707a-ccab-43e7-9287-3c433fbe6d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.395977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152707a-ccab-43e7-9287-3c433fbe6d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.396255 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152707a-ccab-43e7-9287-3c433fbe6d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.397038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152707a-ccab-43e7-9287-3c433fbe6d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.398649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152707a-ccab-43e7-9287-3c433fbe6d7a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.410215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1271/2568] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:18.414187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdacc2-caa1-4841-bbf0-45a39d03f1fe admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.425218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdacc2-caa1-4841-bbf0-45a39d03f1fe admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.429897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b9519-d258-4363-881e-19e6e06f915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.430564 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b9519-d258-4363-881e-19e6e06f915e admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.430849 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b9519-d258-4363-881e-19e6e06f915e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.431126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b9519-d258-4363-881e-19e6e06f915e admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.431854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b9519-d258-4363-881e-19e6e06f915e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.433471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b9519-d258-4363-881e-19e6e06f915e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.444081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1298/2569] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:18.448166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152707a-ccab-43e7-9287-3c433fbe6d7a admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.458093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152707a-ccab-43e7-9287-3c433fbe6d7a admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.463977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ac340-de7b-4731-a027-625de730da57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.464899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ac340-de7b-4731-a027-625de730da57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.465187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ac340-de7b-4731-a027-625de730da57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.465464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ac340-de7b-4731-a027-625de730da57 admin admin] PATH_INFO: `/v3/projects/089df9e029804f739b5e66cc405dda34/users/fa63b28aba2d41ab97cf4ad8d98d9ae0/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.481226 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ac340-de7b-4731-a027-625de730da57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=089df9e029804f739b5e66cc405dda34, user_id=fa63b28aba2d41ab97cf4ad8d98d9ae0, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.483941 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ac340-de7b-4731-a027-625de730da57 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.497310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1272/2570] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:18 2022] PUT /identity/v3/projects/089df9e029804f739b5e66cc405dda34/users/fa63b28aba2d41ab97cf4ad8d98d9ae0/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:18.507660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44501e1b-439c-45ea-b4b9-3cb4f5340ec9 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.507988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44501e1b-439c-45ea-b4b9-3cb4f5340ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.508327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44501e1b-439c-45ea-b4b9-3cb4f5340ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.534824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-44501e1b-439c-45ea-b4b9-3cb4f5340ec9 None None] MFA Rules not processed for user `fa63b28aba2d41ab97cf4ad8d98d9ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:18.536010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-44501e1b-439c-45ea-b4b9-3cb4f5340ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:18.601163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1299/2571] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:18.605709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ac340-de7b-4731-a027-625de730da57 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.635869 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:18.670292 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ac340-de7b-4731-a027-625de730da57 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.675068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefa4d5-3a3f-4776-8ba0-7406ce767ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.675722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefa4d5-3a3f-4776-8ba0-7406ce767ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.676006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefa4d5-3a3f-4776-8ba0-7406ce767ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.676285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefa4d5-3a3f-4776-8ba0-7406ce767ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.677002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefa4d5-3a3f-4776-8ba0-7406ce767ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.678997 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefa4d5-3a3f-4776-8ba0-7406ce767ca4 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.707830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1273/2572] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:18 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:18.712141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44501e1b-439c-45ea-b4b9-3cb4f5340ec9 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.729613 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44501e1b-439c-45ea-b4b9-3cb4f5340ec9 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.755209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:18.792689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3819543-50ca-419b-bafb-744fd172900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.793388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3819543-50ca-419b-bafb-744fd172900e admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.793674 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3819543-50ca-419b-bafb-744fd172900e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.793953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3819543-50ca-419b-bafb-744fd172900e admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.794647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3819543-50ca-419b-bafb-744fd172900e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.796584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3819543-50ca-419b-bafb-744fd172900e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.821174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1300/2573] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:18 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:18.822435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefa4d5-3a3f-4776-8ba0-7406ce767ca4 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.837231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefa4d5-3a3f-4776-8ba0-7406ce767ca4 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.842014 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cac3f-b09f-4125-8d22-50a4dd2e0ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.842673 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cac3f-b09f-4125-8d22-50a4dd2e0ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.843010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cac3f-b09f-4125-8d22-50a4dd2e0ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.843278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cac3f-b09f-4125-8d22-50a4dd2e0ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.844006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cac3f-b09f-4125-8d22-50a4dd2e0ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.845645 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cac3f-b09f-4125-8d22-50a4dd2e0ff2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.858080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1274/2574] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:18.863183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3819543-50ca-419b-bafb-744fd172900e admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.874260 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3819543-50ca-419b-bafb-744fd172900e admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.879306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a5e8d-1888-47b6-b346-43c2b7d9e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.880449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a5e8d-1888-47b6-b346-43c2b7d9e3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.880756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a5e8d-1888-47b6-b346-43c2b7d9e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.881183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a5e8d-1888-47b6-b346-43c2b7d9e3b8 admin admin] PATH_INFO: `/v3/projects/77b71898479b4f759ddb2c07693582a4/users/3c73197d20354b5199df475da33da547/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.897193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a5e8d-1888-47b6-b346-43c2b7d9e3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b71898479b4f759ddb2c07693582a4, user_id=3c73197d20354b5199df475da33da547, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.899806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a5e8d-1888-47b6-b346-43c2b7d9e3b8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.913407 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1301/2575] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:18 2022] PUT /identity/v3/projects/77b71898479b4f759ddb2c07693582a4/users/3c73197d20354b5199df475da33da547/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:18.917417 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cac3f-b09f-4125-8d22-50a4dd2e0ff2 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.927452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cac3f-b09f-4125-8d22-50a4dd2e0ff2 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.982628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2ff40-ac17-4f95-94f3-d9be1619df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.983289 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2ff40-ac17-4f95-94f3-d9be1619df63 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.983574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2ff40-ac17-4f95-94f3-d9be1619df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.983850 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2ff40-ac17-4f95-94f3-d9be1619df63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.984639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2ff40-ac17-4f95-94f3-d9be1619df63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.986284 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2ff40-ac17-4f95-94f3-d9be1619df63 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.996098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1275/2576] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:19.000342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a5e8d-1888-47b6-b346-43c2b7d9e3b8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:19.011472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a5e8d-1888-47b6-b346-43c2b7d9e3b8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:19.016099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0b586-5839-40dc-b050-882499de97db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:19.017012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0b586-5839-40dc-b050-882499de97db admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.017297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0b586-5839-40dc-b050-882499de97db admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.017576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0b586-5839-40dc-b050-882499de97db admin admin] PATH_INFO: `/v3/projects/77b71898479b4f759ddb2c07693582a4/users/3c73197d20354b5199df475da33da547/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.020488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0b586-5839-40dc-b050-882499de97db admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b71898479b4f759ddb2c07693582a4, user_id=3c73197d20354b5199df475da33da547, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:19.023198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0b586-5839-40dc-b050-882499de97db admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:19.034091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1302/2577] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:18 2022] PUT /identity/v3/projects/77b71898479b4f759ddb2c07693582a4/users/3c73197d20354b5199df475da33da547/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:19.045622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72968b0-0294-4ec9-b7fb-7aa34a435562 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.045915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72968b0-0294-4ec9-b7fb-7aa34a435562 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.046196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72968b0-0294-4ec9-b7fb-7aa34a435562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.072348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d72968b0-0294-4ec9-b7fb-7aa34a435562 None None] MFA Rules not processed for user `3c73197d20354b5199df475da33da547`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:19.073590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d72968b0-0294-4ec9-b7fb-7aa34a435562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:19.140019 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1276/2578] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:19.152731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0b586-5839-40dc-b050-882499de97db admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:19.154585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b5f0b586-5839-40dc-b050-882499de97db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:19.216495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0b586-5839-40dc-b050-882499de97db admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:19.220169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696af9f-3a43-46a4-9f86-80a4a2a33081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:19.220841 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696af9f-3a43-46a4-9f86-80a4a2a33081 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.221164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696af9f-3a43-46a4-9f86-80a4a2a33081 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.221444 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696af9f-3a43-46a4-9f86-80a4a2a33081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.230380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696af9f-3a43-46a4-9f86-80a4a2a33081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:19.231816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696af9f-3a43-46a4-9f86-80a4a2a33081 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:19.241312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1303/2579] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:37:19.286398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72968b0-0294-4ec9-b7fb-7aa34a435562 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:19.287911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d72968b0-0294-4ec9-b7fb-7aa34a435562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:19.299724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72968b0-0294-4ec9-b7fb-7aa34a435562 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:19.303241 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947acebc-b55c-4a51-bb40-b523df861169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:19.303870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947acebc-b55c-4a51-bb40-b523df861169 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.304149 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947acebc-b55c-4a51-bb40-b523df861169 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.304421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947acebc-b55c-4a51-bb40-b523df861169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.323779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:19.362906 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947acebc-b55c-4a51-bb40-b523df861169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:19.364442 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947acebc-b55c-4a51-bb40-b523df861169 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:19.373553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1277/2580] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:19 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:22.670301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696af9f-3a43-46a4-9f86-80a4a2a33081 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.735160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696af9f-3a43-46a4-9f86-80a4a2a33081 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.740135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.741055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.741340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.741617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] PATH_INFO: `/v3/users/fa63b28aba2d41ab97cf4ad8d98d9ae0` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.742863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa63b28aba2d41ab97cf4ad8d98d9ae0)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.744828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.770461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:22.796506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa63b28aba2d41ab97cf4ad8d98d9ae0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:22.809880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa63b28aba2d41ab97cf4ad8d98d9ae0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:22.817234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa63b28aba2d41ab97cf4ad8d98d9ae0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:22.817892 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa63b28aba2d41ab97cf4ad8d98d9ae0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:22.818535 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa63b28aba2d41ab97cf4ad8d98d9ae0'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:22.831591 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1304/2581] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:22 2022] DELETE /identity/v3/users/fa63b28aba2d41ab97cf4ad8d98d9ae0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:22.836073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5f22-2fc1-44e6-b0da-a4c7a2b8a6f8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.836520 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5f22-2fc1-44e6-b0da-a4c7a2b8a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.836637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5f22-2fc1-44e6-b0da-a4c7a2b8a6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.854208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-220d5f22-2fc1-44e6-b0da-a4c7a2b8a6f8 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:22.855577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-220d5f22-2fc1-44e6-b0da-a4c7a2b8a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:22.917435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1278/2582] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:22.927545 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.928972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:22.985155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed0119-99b9-4eec-af40-7073834a4716 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.989597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1d9f1-f364-48c6-b71a-7857e4189a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.990248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1d9f1-f364-48c6-b71a-7857e4189a17 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.990535 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1d9f1-f364-48c6-b71a-7857e4189a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.990815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1d9f1-f364-48c6-b71a-7857e4189a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.000099 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1d9f1-f364-48c6-b71a-7857e4189a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.001557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1d9f1-f364-48c6-b71a-7857e4189a17 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.010956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1305/2583] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:23.275030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d5f22-2fc1-44e6-b0da-a4c7a2b8a6f8 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.277525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-220d5f22-2fc1-44e6-b0da-a4c7a2b8a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:23.292918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d5f22-2fc1-44e6-b0da-a4c7a2b8a6f8 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.297810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.298688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.298964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.301121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] PATH_INFO: `/v3/projects/089df9e029804f739b5e66cc405dda34` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.303161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] RBAC: Authorizing `identity:delete_project(project_id=089df9e029804f739b5e66cc405dda34)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.305211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.357522 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '089df9e029804f739b5e66cc405dda34', 'request_id': 'req-e6b1a194-93e4-42c6-994b-237e276209af'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.363801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '089df9e029804f739b5e66cc405dda34', 'request_id': 'req-e6b1a194-93e4-42c6-994b-237e276209af'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.371465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '089df9e029804f739b5e66cc405dda34', 'request_id': 'req-e6b1a194-93e4-42c6-994b-237e276209af'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.384146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] The token cache is being invalidate because project 089df9e029804f739b5e66cc405dda34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:23.384440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.385124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.386689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1279/2584] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:23 2022] DELETE /identity/v3/projects/089df9e029804f739b5e66cc405dda34 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:23.390723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1d9f1-f364-48c6-b71a-7857e4189a17 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.400523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1d9f1-f364-48c6-b71a-7857e4189a17 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.460277 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.461057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.461329 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.461609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] PATH_INFO: `/v3/users/3c73197d20354b5199df475da33da547` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.463091 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c73197d20354b5199df475da33da547)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.465178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.510968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:23.512811 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c73197d20354b5199df475da33da547'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.526827 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c73197d20354b5199df475da33da547'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.534573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c73197d20354b5199df475da33da547'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.535201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c73197d20354b5199df475da33da547'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.535853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c73197d20354b5199df475da33da547'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.548004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1306/2585] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:23 2022] DELETE /identity/v3/users/3c73197d20354b5199df475da33da547 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:23.708152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.709605 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:23.722950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1a194-93e4-42c6-994b-237e276209af admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.767181 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:23.786519 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.787425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.787708 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.787984 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] PATH_INFO: `/v3/users/eeb468d8929e4a3696bc180a565a030b` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.789201 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb468d8929e4a3696bc180a565a030b)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.791381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.841667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb468d8929e4a3696bc180a565a030b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.843131 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb468d8929e4a3696bc180a565a030b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.851903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb468d8929e4a3696bc180a565a030b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.852653 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb468d8929e4a3696bc180a565a030b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.866667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-54cccabf-9cc4-4d42-b2bc-ad1870ca34e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb468d8929e4a3696bc180a565a030b'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:23.868686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.871310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:23.881150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1280/2586] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:23 2022] DELETE /identity/v3/users/eeb468d8929e4a3696bc180a565a030b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:23.885931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22615eb-a994-4494-a757-2a2ef3a19660 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.886297 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22615eb-a994-4494-a757-2a2ef3a19660 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.886629 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22615eb-a994-4494-a757-2a2ef3a19660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.888469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260f06a-948e-487b-b8a4-d71a02ac422d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.909625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e22615eb-a994-4494-a757-2a2ef3a19660 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:23.910915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e22615eb-a994-4494-a757-2a2ef3a19660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:23.952571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.953556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.953837 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.954113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] PATH_INFO: `/v3/projects/77b71898479b4f759ddb2c07693582a4` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.955800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b71898479b4f759ddb2c07693582a4)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.958153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.969915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1281/2587] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:23.980294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22615eb-a994-4494-a757-2a2ef3a19660 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.981728 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e22615eb-a994-4494-a757-2a2ef3a19660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:24.008239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b71898479b4f759ddb2c07693582a4', 'request_id': 'req-ec750900-e398-44db-8f91-797396d60ce2'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.014756 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b71898479b4f759ddb2c07693582a4', 'request_id': 'req-ec750900-e398-44db-8f91-797396d60ce2'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.021216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b71898479b4f759ddb2c07693582a4', 'request_id': 'req-ec750900-e398-44db-8f91-797396d60ce2'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.034874 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] The token cache is being invalidate because project 77b71898479b4f759ddb2c07693582a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:24.035161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.035779 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec750900-e398-44db-8f91-797396d60ce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.038223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1307/2588] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:23 2022] DELETE /identity/v3/projects/77b71898479b4f759ddb2c07693582a4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:24.042949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec2733-659b-4c5c-b860-bdad1be4f2ad None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.043242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec2733-659b-4c5c-b860-bdad1be4f2ad None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.043361 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22615eb-a994-4494-a757-2a2ef3a19660 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.043529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec2733-659b-4c5c-b860-bdad1be4f2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.059424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.062115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-16ec2733-659b-4c5c-b860-bdad1be4f2ad None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:24.063323 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-16ec2733-659b-4c5c-b860-bdad1be4f2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:24.094190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d80102-44a1-4186-a4ca-0636e0aee980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.094832 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d80102-44a1-4186-a4ca-0636e0aee980 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.095119 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d80102-44a1-4186-a4ca-0636e0aee980 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.095386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d80102-44a1-4186-a4ca-0636e0aee980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.097958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-95d80102-44a1-4186-a4ca-0636e0aee980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:24.114596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.129352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1308/2589] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:24.136914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47ee98-8f17-4ec7-95c3-0e6d44e71f70 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.137214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47ee98-8f17-4ec7-95c3-0e6d44e71f70 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.137616 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47ee98-8f17-4ec7-95c3-0e6d44e71f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.157063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4f47ee98-8f17-4ec7-95c3-0e6d44e71f70 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:24.158185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4f47ee98-8f17-4ec7-95c3-0e6d44e71f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:24.163017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d80102-44a1-4186-a4ca-0636e0aee980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.164467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d80102-44a1-4186-a4ca-0636e0aee980 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.166686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1309/2590] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:24.170659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47ee98-8f17-4ec7-95c3-0e6d44e71f70 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.174848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1282/2591] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:37:24.182227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47ee98-8f17-4ec7-95c3-0e6d44e71f70 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.187490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c787529-9242-46ff-bfbb-c1e861e1b4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.188195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c787529-9242-46ff-bfbb-c1e861e1b4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.188482 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c787529-9242-46ff-bfbb-c1e861e1b4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.188815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c787529-9242-46ff-bfbb-c1e861e1b4b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.189581 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c787529-9242-46ff-bfbb-c1e861e1b4b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:24.189928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c787529-9242-46ff-bfbb-c1e861e1b4b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.191670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c787529-9242-46ff-bfbb-c1e861e1b4b8 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.208902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1310/2592] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:24.210695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d80102-44a1-4186-a4ca-0636e0aee980 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.220533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d80102-44a1-4186-a4ca-0636e0aee980 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.226287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfd73f-312a-4426-a55e-d635f004b76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.227082 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfd73f-312a-4426-a55e-d635f004b76e admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.227373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfd73f-312a-4426-a55e-d635f004b76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.227652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfd73f-312a-4426-a55e-d635f004b76e admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.228384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfd73f-312a-4426-a55e-d635f004b76e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.230360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfd73f-312a-4426-a55e-d635f004b76e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.262244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1283/2593] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:24.265597 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c787529-9242-46ff-bfbb-c1e861e1b4b8 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.286029 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c787529-9242-46ff-bfbb-c1e861e1b4b8 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.358016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c1046-3cbb-437b-bfd4-e2a65d35948c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.358711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c1046-3cbb-437b-bfd4-e2a65d35948c admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.359005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c1046-3cbb-437b-bfd4-e2a65d35948c admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.359283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c1046-3cbb-437b-bfd4-e2a65d35948c admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.360013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c1046-3cbb-437b-bfd4-e2a65d35948c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.362203 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c1046-3cbb-437b-bfd4-e2a65d35948c admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.384909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1311/2594] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:24.388401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfd73f-312a-4426-a55e-d635f004b76e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.399534 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfd73f-312a-4426-a55e-d635f004b76e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.404431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4078a45-9bb2-464a-add7-2d2c1c6b463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.405121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4078a45-9bb2-464a-add7-2d2c1c6b463d admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.405404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4078a45-9bb2-464a-add7-2d2c1c6b463d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.405675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4078a45-9bb2-464a-add7-2d2c1c6b463d admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.406402 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4078a45-9bb2-464a-add7-2d2c1c6b463d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.408071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4078a45-9bb2-464a-add7-2d2c1c6b463d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.422035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1284/2595] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:24.423318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c1046-3cbb-437b-bfd4-e2a65d35948c admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.433571 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c1046-3cbb-437b-bfd4-e2a65d35948c admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.438286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e179c7-9d83-4870-b81a-34b24051cde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.438953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e179c7-9d83-4870-b81a-34b24051cde9 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.439241 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e179c7-9d83-4870-b81a-34b24051cde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.439520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e179c7-9d83-4870-b81a-34b24051cde9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.440991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e179c7-9d83-4870-b81a-34b24051cde9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.442577 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e179c7-9d83-4870-b81a-34b24051cde9 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.452609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1312/2596] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:24.458152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4078a45-9bb2-464a-add7-2d2c1c6b463d admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.468219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4078a45-9bb2-464a-add7-2d2c1c6b463d admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.472431 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e179c7-9d83-4870-b81a-34b24051cde9 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.474295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-74e179c7-9d83-4870-b81a-34b24051cde9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:24.478036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc554f-8c62-4f09-86aa-090b7b2e7486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.478953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc554f-8c62-4f09-86aa-090b7b2e7486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.479237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc554f-8c62-4f09-86aa-090b7b2e7486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.479497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc554f-8c62-4f09-86aa-090b7b2e7486 admin admin] PATH_INFO: `/v3/projects/fbb48f6cb69a47e6922eda90627d1dc8/users/43341db76c4a42c1bdff59da3c1c0713/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.487928 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e179c7-9d83-4870-b81a-34b24051cde9 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.493096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.493963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.494244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.494515 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] PATH_INFO: `/v3/projects/86083144dbcd42c491a8086e3800a6cd` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.495712 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86083144dbcd42c491a8086e3800a6cd)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.497819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.498311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc554f-8c62-4f09-86aa-090b7b2e7486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb48f6cb69a47e6922eda90627d1dc8, user_id=43341db76c4a42c1bdff59da3c1c0713, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.501251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc554f-8c62-4f09-86aa-090b7b2e7486 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.517600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1285/2597] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:24 2022] PUT /identity/v3/projects/fbb48f6cb69a47e6922eda90627d1dc8/users/43341db76c4a42c1bdff59da3c1c0713/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:24.530813 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862393f-3802-48cf-9f6c-cda11d03c1fc None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.531259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862393f-3802-48cf-9f6c-cda11d03c1fc None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.531545 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862393f-3802-48cf-9f6c-cda11d03c1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.548788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86083144dbcd42c491a8086e3800a6cd', 'request_id': 'req-ec09599a-c8db-424e-ba79-528af6f91b53'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.555717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86083144dbcd42c491a8086e3800a6cd', 'request_id': 'req-ec09599a-c8db-424e-ba79-528af6f91b53'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.557710 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a862393f-3802-48cf-9f6c-cda11d03c1fc None None] MFA Rules not processed for user `43341db76c4a42c1bdff59da3c1c0713`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:24.558927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a862393f-3802-48cf-9f6c-cda11d03c1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:24.561249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86083144dbcd42c491a8086e3800a6cd', 'request_id': 'req-ec09599a-c8db-424e-ba79-528af6f91b53'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.575252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] The token cache is being invalidate because project 86083144dbcd42c491a8086e3800a6cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:24.575542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.576287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ec09599a-c8db-424e-ba79-528af6f91b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:24.577893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1313/2598] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:24 2022] DELETE /identity/v3/projects/86083144dbcd42c491a8086e3800a6cd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:24.583739 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf5cff-6776-444d-a3a7-3542ab47f6d6 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.584021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf5cff-6776-444d-a3a7-3542ab47f6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.584322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf5cff-6776-444d-a3a7-3542ab47f6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.607660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-e9cf5cff-6776-444d-a3a7-3542ab47f6d6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:24.608939 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e9cf5cff-6776-444d-a3a7-3542ab47f6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:24.627500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1286/2599] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:24.632555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862393f-3802-48cf-9f6c-cda11d03c1fc None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.634107 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a862393f-3802-48cf-9f6c-cda11d03c1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:24.639093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.640490 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.675240 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1314/2600] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:24.683350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a34d15-8813-4041-81a9-7ec64fea93d6 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.683642 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a34d15-8813-4041-81a9-7ec64fea93d6 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.683920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a34d15-8813-4041-81a9-7ec64fea93d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.704972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-83a34d15-8813-4041-81a9-7ec64fea93d6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:24.706229 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-83a34d15-8813-4041-81a9-7ec64fea93d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:24.709821 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862393f-3802-48cf-9f6c-cda11d03c1fc None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.714187 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1315/2601] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:24.714824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ede57-c334-40e1-bd81-0a601d90bf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.715626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ede57-c334-40e1-bd81-0a601d90bf6e admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.715931 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ede57-c334-40e1-bd81-0a601d90bf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.716211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ede57-c334-40e1-bd81-0a601d90bf6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.716936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ede57-c334-40e1-bd81-0a601d90bf6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.718289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a34d15-8813-4041-81a9-7ec64fea93d6 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.719122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ede57-c334-40e1-bd81-0a601d90bf6e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.727986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a34d15-8813-4041-81a9-7ec64fea93d6 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.732694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb0b52-22ac-41ac-a343-a2d13996be1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.733389 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb0b52-22ac-41ac-a343-a2d13996be1b admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.733677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb0b52-22ac-41ac-a343-a2d13996be1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.733957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb0b52-22ac-41ac-a343-a2d13996be1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.734707 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb0b52-22ac-41ac-a343-a2d13996be1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:24.735020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb0b52-22ac-41ac-a343-a2d13996be1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.737505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb0b52-22ac-41ac-a343-a2d13996be1b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.746772 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1287/2602] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:24.750657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ede57-c334-40e1-bd81-0a601d90bf6e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.753954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1316/2603] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:24.754865 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb0b52-22ac-41ac-a343-a2d13996be1b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.765151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb0b52-22ac-41ac-a343-a2d13996be1b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.816804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ede57-c334-40e1-bd81-0a601d90bf6e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.821955 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cf44b-6b2c-4749-aa18-c1c563d04c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.822633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cf44b-6b2c-4749-aa18-c1c563d04c27 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.822915 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cf44b-6b2c-4749-aa18-c1c563d04c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.823250 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cf44b-6b2c-4749-aa18-c1c563d04c27 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.823937 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cf44b-6b2c-4749-aa18-c1c563d04c27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.825923 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cf44b-6b2c-4749-aa18-c1c563d04c27 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.826285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eea341-8559-4fd7-941a-755a6c8c20b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.826949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eea341-8559-4fd7-941a-755a6c8c20b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.827237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eea341-8559-4fd7-941a-755a6c8c20b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.827520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eea341-8559-4fd7-941a-755a6c8c20b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.828361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eea341-8559-4fd7-941a-755a6c8c20b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.830281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eea341-8559-4fd7-941a-755a6c8c20b4 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.849109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1288/2604] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:24.851024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cf44b-6b2c-4749-aa18-c1c563d04c27 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.857981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1317/2605] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:24.862578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eea341-8559-4fd7-941a-755a6c8c20b4 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.866382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cf44b-6b2c-4749-aa18-c1c563d04c27 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.923521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c820b4-5ba2-45bf-9eec-7c4f4c032c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.923726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eea341-8559-4fd7-941a-755a6c8c20b4 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.924717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c820b4-5ba2-45bf-9eec-7c4f4c032c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.925203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c820b4-5ba2-45bf-9eec-7c4f4c032c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.925203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c820b4-5ba2-45bf-9eec-7c4f4c032c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.925961 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c820b4-5ba2-45bf-9eec-7c4f4c032c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.927662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c820b4-5ba2-45bf-9eec-7c4f4c032c46 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.929213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393d628-dbc9-425d-8e16-8f0ba85616bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.929884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393d628-dbc9-425d-8e16-8f0ba85616bf admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.930169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393d628-dbc9-425d-8e16-8f0ba85616bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.930449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393d628-dbc9-425d-8e16-8f0ba85616bf admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.931141 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393d628-dbc9-425d-8e16-8f0ba85616bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.933089 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393d628-dbc9-425d-8e16-8f0ba85616bf admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.937597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1289/2606] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:24.942502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c820b4-5ba2-45bf-9eec-7c4f4c032c46 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.953539 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c820b4-5ba2-45bf-9eec-7c4f4c032c46 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.954825 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1318/2607] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:24 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:24.958367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f28f9-dd98-4344-9f04-ca681487e066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.959018 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393d628-dbc9-425d-8e16-8f0ba85616bf admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.959258 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f28f9-dd98-4344-9f04-ca681487e066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.959567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f28f9-dd98-4344-9f04-ca681487e066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.959840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f28f9-dd98-4344-9f04-ca681487e066 admin admin] PATH_INFO: `/v3/projects/013bb5ff1d7043e5962a4976f3329e6c/users/62a5d106ff154118b5f95a4a0c2a5fac/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.968799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393d628-dbc9-425d-8e16-8f0ba85616bf admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.972243 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.975343 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1892e-b819-4448-bd5f-a9884eea44c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.976314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1892e-b819-4448-bd5f-a9884eea44c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.976768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1892e-b819-4448-bd5f-a9884eea44c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.977164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1892e-b819-4448-bd5f-a9884eea44c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.977419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f28f9-dd98-4344-9f04-ca681487e066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013bb5ff1d7043e5962a4976f3329e6c, user_id=62a5d106ff154118b5f95a4a0c2a5fac, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.978125 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1892e-b819-4448-bd5f-a9884eea44c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.979720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1892e-b819-4448-bd5f-a9884eea44c4 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.980245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f28f9-dd98-4344-9f04-ca681487e066 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.990558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1319/2608] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:24.994331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1290/2609] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:24 2022] PUT /identity/v3/projects/013bb5ff1d7043e5962a4976f3329e6c/users/62a5d106ff154118b5f95a4a0c2a5fac/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:24.994545 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1892e-b819-4448-bd5f-a9884eea44c4 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.998127 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f28f9-dd98-4344-9f04-ca681487e066 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.004600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1892e-b819-4448-bd5f-a9884eea44c4 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.022913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.061091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f28f9-dd98-4344-9f04-ca681487e066 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.064500 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c3844-48c2-4e59-99e6-edfba08c7b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.065771 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c3844-48c2-4e59-99e6-edfba08c7b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.066086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c3844-48c2-4e59-99e6-edfba08c7b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.066789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c3844-48c2-4e59-99e6-edfba08c7b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.066884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757d791-ef79-4dda-acc6-2988ad75e3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.067090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c3844-48c2-4e59-99e6-edfba08c7b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.067264 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757d791-ef79-4dda-acc6-2988ad75e3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.067734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757d791-ef79-4dda-acc6-2988ad75e3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.068111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757d791-ef79-4dda-acc6-2988ad75e3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.068962 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757d791-ef79-4dda-acc6-2988ad75e3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.069071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c3844-48c2-4e59-99e6-edfba08c7b19 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.070584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757d791-ef79-4dda-acc6-2988ad75e3bd admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.079043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1320/2610] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:25.081529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1291/2611] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:25.085678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c3844-48c2-4e59-99e6-edfba08c7b19 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.085816 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757d791-ef79-4dda-acc6-2988ad75e3bd admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.095651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c3844-48c2-4e59-99e6-edfba08c7b19 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.095846 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757d791-ef79-4dda-acc6-2988ad75e3bd admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.100572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9e706-34fe-4a47-ab7d-0bb64226d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.100707 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19ca59-a11f-4e28-a032-d838f644cbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.101468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9e706-34fe-4a47-ab7d-0bb64226d083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.101597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ca59-a11f-4e28-a032-d838f644cbb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.101753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9e706-34fe-4a47-ab7d-0bb64226d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.101849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ca59-a11f-4e28-a032-d838f644cbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.102031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9e706-34fe-4a47-ab7d-0bb64226d083 admin admin] PATH_INFO: `/v3/projects/d0edf57bf68147baa4cc35e69652687d/users/d2c670495e71403c97d251118f55edfd/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.102130 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ca59-a11f-4e28-a032-d838f644cbb3 admin admin] PATH_INFO: `/v3/projects/013bb5ff1d7043e5962a4976f3329e6c/users/62a5d106ff154118b5f95a4a0c2a5fac/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.104080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19ca59-a11f-4e28-a032-d838f644cbb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013bb5ff1d7043e5962a4976f3329e6c, user_id=62a5d106ff154118b5f95a4a0c2a5fac, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.106841 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19ca59-a11f-4e28-a032-d838f644cbb3 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.117945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1292/2612] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:25 2022] PUT /identity/v3/projects/013bb5ff1d7043e5962a4976f3329e6c/users/62a5d106ff154118b5f95a4a0c2a5fac/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:25.119637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9e706-34fe-4a47-ab7d-0bb64226d083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0edf57bf68147baa4cc35e69652687d, user_id=d2c670495e71403c97d251118f55edfd, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.122385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9e706-34fe-4a47-ab7d-0bb64226d083 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.128579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51550e-3fa9-44a7-84e7-6c2a02fe40b7 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.128935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51550e-3fa9-44a7-84e7-6c2a02fe40b7 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.129254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51550e-3fa9-44a7-84e7-6c2a02fe40b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.136298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1321/2613] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:25 2022] PUT /identity/v3/projects/d0edf57bf68147baa4cc35e69652687d/users/d2c670495e71403c97d251118f55edfd/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:25.148487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff44e6-850f-4aa0-903d-b3d0afaa71d9 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.148782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff44e6-850f-4aa0-903d-b3d0afaa71d9 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.149090 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff44e6-850f-4aa0-903d-b3d0afaa71d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.156738 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-9d51550e-3fa9-44a7-84e7-6c2a02fe40b7 None None] MFA Rules not processed for user `62a5d106ff154118b5f95a4a0c2a5fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:25.157987 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-9d51550e-3fa9-44a7-84e7-6c2a02fe40b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:25.175681 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-bdff44e6-850f-4aa0-903d-b3d0afaa71d9 None None] MFA Rules not processed for user `d2c670495e71403c97d251118f55edfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:25.176819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-bdff44e6-850f-4aa0-903d-b3d0afaa71d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:25.227898 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1293/2614] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:25.238519 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51550e-3fa9-44a7-84e7-6c2a02fe40b7 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.239896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-9d51550e-3fa9-44a7-84e7-6c2a02fe40b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:25.242759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1322/2615] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:25.247168 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff44e6-850f-4aa0-903d-b3d0afaa71d9 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.297569 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51550e-3fa9-44a7-84e7-6c2a02fe40b7 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.301412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4c47a-5c39-465f-a4e7-04f8d23214d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.302066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4c47a-5c39-465f-a4e7-04f8d23214d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.302348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4c47a-5c39-465f-a4e7-04f8d23214d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.302622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4c47a-5c39-465f-a4e7-04f8d23214d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.308617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff44e6-850f-4aa0-903d-b3d0afaa71d9 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.312428 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4c47a-5c39-465f-a4e7-04f8d23214d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.313414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07010498-f277-4add-8718-9fcfd7ee6228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.313900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4c47a-5c39-465f-a4e7-04f8d23214d6 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.314070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07010498-f277-4add-8718-9fcfd7ee6228 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.314357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07010498-f277-4add-8718-9fcfd7ee6228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.314635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07010498-f277-4add-8718-9fcfd7ee6228 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.315824 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07010498-f277-4add-8718-9fcfd7ee6228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.317753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07010498-f277-4add-8718-9fcfd7ee6228 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.319468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.326590 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1294/2616] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:37:25.346789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1323/2617] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:25 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:25.347967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4c47a-5c39-465f-a4e7-04f8d23214d6 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.357983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4c47a-5c39-465f-a4e7-04f8d23214d6 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.422169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2acef-8861-47d9-816b-dea017bbc9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.422837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2acef-8861-47d9-816b-dea017bbc9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.423122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2acef-8861-47d9-816b-dea017bbc9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.423397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2acef-8861-47d9-816b-dea017bbc9b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.424624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2acef-8861-47d9-816b-dea017bbc9b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.426819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2acef-8861-47d9-816b-dea017bbc9b5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.451398 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1295/2618] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:25 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:25.452887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07010498-f277-4add-8718-9fcfd7ee6228 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.467997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07010498-f277-4add-8718-9fcfd7ee6228 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.472804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384abc82-20f6-4711-ac06-66f7efd34d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.473439 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384abc82-20f6-4711-ac06-66f7efd34d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.473727 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384abc82-20f6-4711-ac06-66f7efd34d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.474006 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384abc82-20f6-4711-ac06-66f7efd34d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.474747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384abc82-20f6-4711-ac06-66f7efd34d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.476344 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384abc82-20f6-4711-ac06-66f7efd34d61 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.477883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.489833 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1324/2619] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:25.494043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2acef-8861-47d9-816b-dea017bbc9b5 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.504906 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2acef-8861-47d9-816b-dea017bbc9b5 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.509849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb02531-448e-46a8-b0de-ecda60684dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.510771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb02531-448e-46a8-b0de-ecda60684dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.511056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb02531-448e-46a8-b0de-ecda60684dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.511334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb02531-448e-46a8-b0de-ecda60684dbf admin admin] PATH_INFO: `/v3/projects/d7a05a26a0c34a8bbd077944bd91fe1c/users/f730b568533442a1b0643e4e4f8f3505/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.528239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb02531-448e-46a8-b0de-ecda60684dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a05a26a0c34a8bbd077944bd91fe1c, user_id=f730b568533442a1b0643e4e4f8f3505, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.531046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb02531-448e-46a8-b0de-ecda60684dbf admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.545455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1296/2620] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:25 2022] PUT /identity/v3/projects/d7a05a26a0c34a8bbd077944bd91fe1c/users/f730b568533442a1b0643e4e4f8f3505/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:25.549764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384abc82-20f6-4711-ac06-66f7efd34d61 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.561109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384abc82-20f6-4711-ac06-66f7efd34d61 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.626606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53544102-3985-4186-bb1b-cc2963ca8042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.628045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53544102-3985-4186-bb1b-cc2963ca8042 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.628550 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53544102-3985-4186-bb1b-cc2963ca8042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.629051 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53544102-3985-4186-bb1b-cc2963ca8042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.629853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53544102-3985-4186-bb1b-cc2963ca8042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.631448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53544102-3985-4186-bb1b-cc2963ca8042 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.641968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1325/2621] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:25.646310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb02531-448e-46a8-b0de-ecda60684dbf admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.656947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb02531-448e-46a8-b0de-ecda60684dbf admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.662192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c48ff6-ddb8-49ec-b0cc-ff09dd805156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.663276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48ff6-ddb8-49ec-b0cc-ff09dd805156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.663276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48ff6-ddb8-49ec-b0cc-ff09dd805156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.663479 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48ff6-ddb8-49ec-b0cc-ff09dd805156 admin admin] PATH_INFO: `/v3/projects/d7a05a26a0c34a8bbd077944bd91fe1c/users/f730b568533442a1b0643e4e4f8f3505/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.665559 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c48ff6-ddb8-49ec-b0cc-ff09dd805156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a05a26a0c34a8bbd077944bd91fe1c, user_id=f730b568533442a1b0643e4e4f8f3505, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.668148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c48ff6-ddb8-49ec-b0cc-ff09dd805156 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.680407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1297/2622] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:25 2022] PUT /identity/v3/projects/d7a05a26a0c34a8bbd077944bd91fe1c/users/f730b568533442a1b0643e4e4f8f3505/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:25.690968 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e03908-eeb4-4db7-8532-4a1739689ba0 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.691339 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e03908-eeb4-4db7-8532-4a1739689ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.691690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e03908-eeb4-4db7-8532-4a1739689ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.722556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f5e03908-eeb4-4db7-8532-4a1739689ba0 None None] MFA Rules not processed for user `f730b568533442a1b0643e4e4f8f3505`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:25.723768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f5e03908-eeb4-4db7-8532-4a1739689ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:25.743734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c48ff6-ddb8-49ec-b0cc-ff09dd805156 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.794871 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1326/2623] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:25.806263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e03908-eeb4-4db7-8532-4a1739689ba0 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.808068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f5e03908-eeb4-4db7-8532-4a1739689ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:25.810935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c48ff6-ddb8-49ec-b0cc-ff09dd805156 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.816345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.817245 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.817532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.817808 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] PATH_INFO: `/v3/users/43341db76c4a42c1bdff59da3c1c0713` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.819015 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43341db76c4a42c1bdff59da3c1c0713)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.821010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.866651 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e03908-eeb4-4db7-8532-4a1739689ba0 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.869338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43341db76c4a42c1bdff59da3c1c0713'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:25.870075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43341db76c4a42c1bdff59da3c1c0713'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:25.870269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc314b5e-8e86-4686-854c-78c99c1d80ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.870944 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc314b5e-8e86-4686-854c-78c99c1d80ac service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.871226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc314b5e-8e86-4686-854c-78c99c1d80ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.871501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc314b5e-8e86-4686-854c-78c99c1d80ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.875406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.878337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43341db76c4a42c1bdff59da3c1c0713'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:25.879802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43341db76c4a42c1bdff59da3c1c0713'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:25.881628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc314b5e-8e86-4686-854c-78c99c1d80ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.883055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc314b5e-8e86-4686-854c-78c99c1d80ac service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.892588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1327/2624] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:37:25.895625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43341db76c4a42c1bdff59da3c1c0713'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:25.909884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1298/2625] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:25 2022] DELETE /identity/v3/users/43341db76c4a42c1bdff59da3c1c0713 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:25.914734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93e4ee-ce2e-4a1f-a278-0e0663b817dd None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.915055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93e4ee-ce2e-4a1f-a278-0e0663b817dd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.915299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93e4ee-ce2e-4a1f-a278-0e0663b817dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.935213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6b93e4ee-ce2e-4a1f-a278-0e0663b817dd None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:25.936493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6b93e4ee-ce2e-4a1f-a278-0e0663b817dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:25.950119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:26.007492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1328/2626] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:26.017056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.019172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:26.079224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ae972-059a-4aac-9eb5-177fea0605d3 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.083957 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e5d0-1426-47ea-b710-c01aa6cf8c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.084602 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e5d0-1426-47ea-b710-c01aa6cf8c9d service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.084908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e5d0-1426-47ea-b710-c01aa6cf8c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.085187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e5d0-1426-47ea-b710-c01aa6cf8c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.096146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8e5d0-1426-47ea-b710-c01aa6cf8c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.097765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8e5d0-1426-47ea-b710-c01aa6cf8c9d service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.107774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1299/2627] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:26.325692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93e4ee-ce2e-4a1f-a278-0e0663b817dd None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.327450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6b93e4ee-ce2e-4a1f-a278-0e0663b817dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:26.339648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93e4ee-ce2e-4a1f-a278-0e0663b817dd None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.344182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.345095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.345376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.345650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] PATH_INFO: `/v3/users/d2c670495e71403c97d251118f55edfd` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.346886 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c670495e71403c97d251118f55edfd)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.348797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.407015 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c670495e71403c97d251118f55edfd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.407628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e5d0-1426-47ea-b710-c01aa6cf8c9d service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.410071 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e9b8e5d0-1426-47ea-b710-c01aa6cf8c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:26.420217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c670495e71403c97d251118f55edfd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.430328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c670495e71403c97d251118f55edfd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.431255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c670495e71403c97d251118f55edfd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.431869 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e60e19e0-45cd-4300-a3f9-e7bbb6cd1ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c670495e71403c97d251118f55edfd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.434624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e5d0-1426-47ea-b710-c01aa6cf8c9d service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.440429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.441379 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.441771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.442051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] PATH_INFO: `/v3/projects/fbb48f6cb69a47e6922eda90627d1dc8` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.443308 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb48f6cb69a47e6922eda90627d1dc8)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.445249 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.447292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1329/2628] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:26 2022] DELETE /identity/v3/users/d2c670495e71403c97d251118f55edfd => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:26.451627 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a3cc-4ebb-42af-abcd-4036a65a9167 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.451923 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a3cc-4ebb-42af-abcd-4036a65a9167 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.452205 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a3cc-4ebb-42af-abcd-4036a65a9167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.469914 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-09a9a3cc-4ebb-42af-abcd-4036a65a9167 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:26.471804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-09a9a3cc-4ebb-42af-abcd-4036a65a9167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:26.495613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb48f6cb69a47e6922eda90627d1dc8', 'request_id': 'req-41e65725-51c4-4715-9ae0-b6493f38cf40'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.501784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb48f6cb69a47e6922eda90627d1dc8', 'request_id': 'req-41e65725-51c4-4715-9ae0-b6493f38cf40'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.510026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb48f6cb69a47e6922eda90627d1dc8', 'request_id': 'req-41e65725-51c4-4715-9ae0-b6493f38cf40'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.521823 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] The token cache is being invalidate because project fbb48f6cb69a47e6922eda90627d1dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:26.522136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.522810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.524393 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1300/2629] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:26 2022] DELETE /identity/v3/projects/fbb48f6cb69a47e6922eda90627d1dc8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:26.528692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.530112 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:26.535959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1330/2630] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:26.545607 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a3cc-4ebb-42af-abcd-4036a65a9167 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.546940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-09a9a3cc-4ebb-42af-abcd-4036a65a9167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:26.585725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:26.594006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:26.600287 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e65725-51c4-4715-9ae0-b6493f38cf40 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.607893 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.610976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.611232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.611516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] PATH_INFO: `/v3/users/62a5d106ff154118b5f95a4a0c2a5fac` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.613579 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a5d106ff154118b5f95a4a0c2a5fac)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.614365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a3cc-4ebb-42af-abcd-4036a65a9167 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.615505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.662635 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a5d106ff154118b5f95a4a0c2a5fac'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.663319 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a5d106ff154118b5f95a4a0c2a5fac'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.667578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.670953 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a5d106ff154118b5f95a4a0c2a5fac'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.671677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a5d106ff154118b5f95a4a0c2a5fac'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.677411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.678056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.678056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.681045 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:26.685550 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a5d106ff154118b5f95a4a0c2a5fac'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:26.697643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.698514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.706394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1301/2631] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:26 2022] DELETE /identity/v3/users/62a5d106ff154118b5f95a4a0c2a5fac => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:26.709340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1331/2632] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:37:27.097752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.099766 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.124731 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.126174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.138609 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07f68e-cb04-4155-b1bc-2e43d6785b1b service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.177421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e3e1f-ecca-4316-86cd-aa3d9986b878 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.182488 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.183351 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.183634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.183911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] PATH_INFO: `/v3/projects/d0edf57bf68147baa4cc35e69652687d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.185218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0edf57bf68147baa4cc35e69652687d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.187141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.202750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.203670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.203838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.204117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] PATH_INFO: `/v3/projects/013bb5ff1d7043e5962a4976f3329e6c` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.205895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=013bb5ff1d7043e5962a4976f3329e6c)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.207998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.235928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0edf57bf68147baa4cc35e69652687d', 'request_id': 'req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.242802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0edf57bf68147baa4cc35e69652687d', 'request_id': 'req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.251291 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0edf57bf68147baa4cc35e69652687d', 'request_id': 'req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.257104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '013bb5ff1d7043e5962a4976f3329e6c', 'request_id': 'req-61ae0a82-3af5-4654-b418-b6f3d3210b87'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.263305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] The token cache is being invalidate because project d0edf57bf68147baa4cc35e69652687d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:27.263528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '013bb5ff1d7043e5962a4976f3329e6c', 'request_id': 'req-61ae0a82-3af5-4654-b418-b6f3d3210b87'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.263637 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.264218 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.266208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1302/2633] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:27 2022] DELETE /identity/v3/projects/d0edf57bf68147baa4cc35e69652687d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:27.270664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.271907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '013bb5ff1d7043e5962a4976f3329e6c', 'request_id': 'req-61ae0a82-3af5-4654-b418-b6f3d3210b87'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.272041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.282445 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.285331 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] The token cache is being invalidate because project 013bb5ff1d7043e5962a4976f3329e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:27.285629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.286267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-61ae0a82-3af5-4654-b418-b6f3d3210b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.287836 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1332/2634] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:27 2022] DELETE /identity/v3/projects/013bb5ff1d7043e5962a4976f3329e6c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:27.295115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbafad-edaa-4cb1-b4c6-fd0f1de9cf76 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.295379 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbafad-edaa-4cb1-b4c6-fd0f1de9cf76 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.295658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbafad-edaa-4cb1-b4c6-fd0f1de9cf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.314221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b4bbafad-edaa-4cb1-b4c6-fd0f1de9cf76 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:27.316191 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b4bbafad-edaa-4cb1-b4c6-fd0f1de9cf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.334565 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.342382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11668f3b-1bf3-4e14-a32c-dba9f8a0d928 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.380539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1333/2635] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:27.388433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dced1ba-d48f-458e-85f0-41c02e434abf None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.388726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dced1ba-d48f-458e-85f0-41c02e434abf None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.389068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dced1ba-d48f-458e-85f0-41c02e434abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.399480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.400385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.400668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.400981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] PATH_INFO: `/v3/users/f730b568533442a1b0643e4e4f8f3505` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.402435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f730b568533442a1b0643e4e4f8f3505)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.404368 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.407369 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-7dced1ba-d48f-458e-85f0-41c02e434abf None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:27.408645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-7dced1ba-d48f-458e-85f0-41c02e434abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.416726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1334/2636] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:27.421617 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dced1ba-d48f-458e-85f0-41c02e434abf None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.433180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dced1ba-d48f-458e-85f0-41c02e434abf None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.438463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f68223-60c7-4442-89fa-813e75287348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.439155 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f68223-60c7-4442-89fa-813e75287348 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.439445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f68223-60c7-4442-89fa-813e75287348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.440236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f68223-60c7-4442-89fa-813e75287348 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.441108 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f68223-60c7-4442-89fa-813e75287348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:27.441420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f68223-60c7-4442-89fa-813e75287348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.443017 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f68223-60c7-4442-89fa-813e75287348 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.453356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f730b568533442a1b0643e4e4f8f3505'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.453966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f730b568533442a1b0643e4e4f8f3505'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.454863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1335/2637] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:27.459662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f68223-60c7-4442-89fa-813e75287348 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.463795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f730b568533442a1b0643e4e4f8f3505'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.464523 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f730b568533442a1b0643e4e4f8f3505'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.469689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f68223-60c7-4442-89fa-813e75287348 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.474547 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ae3d1-4431-4e8c-97c9-7e0ff5af4a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.475220 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ae3d1-4431-4e8c-97c9-7e0ff5af4a5e admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.475508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ae3d1-4431-4e8c-97c9-7e0ff5af4a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.475806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ae3d1-4431-4e8c-97c9-7e0ff5af4a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.476488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ae3d1-4431-4e8c-97c9-7e0ff5af4a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.477429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f730b568533442a1b0643e4e4f8f3505'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.478409 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ae3d1-4431-4e8c-97c9-7e0ff5af4a5e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.491065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1303/2638] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:27 2022] DELETE /identity/v3/users/f730b568533442a1b0643e4e4f8f3505 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:27.507489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1336/2639] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:27 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:27.511594 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.513090 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.552217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.590379 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03fc77-66d6-40ed-8da5-65d3b959c342 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.658141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeee88e-859f-4b6f-8fba-e6671914a03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.659634 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeee88e-859f-4b6f-8fba-e6671914a03f admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.660002 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeee88e-859f-4b6f-8fba-e6671914a03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.660288 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeee88e-859f-4b6f-8fba-e6671914a03f admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.661263 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeee88e-859f-4b6f-8fba-e6671914a03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.663310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeee88e-859f-4b6f-8fba-e6671914a03f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.684901 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1304/2640] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:27 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:27.688049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ae3d1-4431-4e8c-97c9-7e0ff5af4a5e admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.698956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ae3d1-4431-4e8c-97c9-7e0ff5af4a5e admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.703956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37428e48-3380-4352-b41f-92639eabddca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.704569 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37428e48-3380-4352-b41f-92639eabddca admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.704917 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37428e48-3380-4352-b41f-92639eabddca admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.705204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37428e48-3380-4352-b41f-92639eabddca admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.706689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37428e48-3380-4352-b41f-92639eabddca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.708299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37428e48-3380-4352-b41f-92639eabddca admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.722621 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1337/2641] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:27.723792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeee88e-859f-4b6f-8fba-e6671914a03f admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.733828 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeee88e-859f-4b6f-8fba-e6671914a03f admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.738702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94146a28-784b-4eaa-bf22-f403849b33ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.739366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94146a28-784b-4eaa-bf22-f403849b33ff admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.739648 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94146a28-784b-4eaa-bf22-f403849b33ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.739922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94146a28-784b-4eaa-bf22-f403849b33ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.740675 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94146a28-784b-4eaa-bf22-f403849b33ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.742327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94146a28-784b-4eaa-bf22-f403849b33ff admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.754164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1305/2642] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:27.758468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37428e48-3380-4352-b41f-92639eabddca admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.773350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37428e48-3380-4352-b41f-92639eabddca admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.780016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09449a32-4b8f-42db-b1c1-70668566db85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.781081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09449a32-4b8f-42db-b1c1-70668566db85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.781486 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09449a32-4b8f-42db-b1c1-70668566db85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.781880 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09449a32-4b8f-42db-b1c1-70668566db85 admin admin] PATH_INFO: `/v3/projects/e7dc97a4abe84effa3a24e933febb468/users/e953b269bc00433381f03983cd575a80/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.789173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94146a28-784b-4eaa-bf22-f403849b33ff admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.790990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-94146a28-784b-4eaa-bf22-f403849b33ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.804185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09449a32-4b8f-42db-b1c1-70668566db85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7dc97a4abe84effa3a24e933febb468, user_id=e953b269bc00433381f03983cd575a80, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.805907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94146a28-784b-4eaa-bf22-f403849b33ff admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.807653 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09449a32-4b8f-42db-b1c1-70668566db85 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.811214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.812211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.812567 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.812946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] PATH_INFO: `/v3/projects/d7a05a26a0c34a8bbd077944bd91fe1c` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.814227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7a05a26a0c34a8bbd077944bd91fe1c)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.817822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.828287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1338/2643] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:27 2022] PUT /identity/v3/projects/e7dc97a4abe84effa3a24e933febb468/users/e953b269bc00433381f03983cd575a80/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:27.839683 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502b816-7c00-458e-89d1-c402ad6d90d8 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.839971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502b816-7c00-458e-89d1-c402ad6d90d8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.840245 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502b816-7c00-458e-89d1-c402ad6d90d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.870265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-9502b816-7c00-458e-89d1-c402ad6d90d8 None None] MFA Rules not processed for user `e953b269bc00433381f03983cd575a80`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:27.871723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9502b816-7c00-458e-89d1-c402ad6d90d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.872762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7a05a26a0c34a8bbd077944bd91fe1c', 'request_id': 'req-f527ef7c-75c5-4391-9e98-41caa901b110'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.879001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7a05a26a0c34a8bbd077944bd91fe1c', 'request_id': 'req-f527ef7c-75c5-4391-9e98-41caa901b110'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.887040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7a05a26a0c34a8bbd077944bd91fe1c', 'request_id': 'req-f527ef7c-75c5-4391-9e98-41caa901b110'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.899297 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] The token cache is being invalidate because project d7a05a26a0c34a8bbd077944bd91fe1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:27.899585 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.900238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f527ef7c-75c5-4391-9e98-41caa901b110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:27.902306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1306/2644] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:27 2022] DELETE /identity/v3/projects/d7a05a26a0c34a8bbd077944bd91fe1c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:27.906641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe73cd7-d666-4df4-b646-d5a45bec8df4 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.906929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe73cd7-d666-4df4-b646-d5a45bec8df4 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.907205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe73cd7-d666-4df4-b646-d5a45bec8df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.924642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-abe73cd7-d666-4df4-b646-d5a45bec8df4 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:27.925908 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-abe73cd7-d666-4df4-b646-d5a45bec8df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.941361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1339/2645] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:27.952106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502b816-7c00-458e-89d1-c402ad6d90d8 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.953757 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9502b816-7c00-458e-89d1-c402ad6d90d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.966894 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.997040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1307/2646] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:27.998610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31f5f-957b-472f-a587-0024351b72d2 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.998896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31f5f-957b-472f-a587-0024351b72d2 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.999228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31f5f-957b-472f-a587-0024351b72d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.000964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.020078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-4fd31f5f-957b-472f-a587-0024351b72d2 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:28.020517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502b816-7c00-458e-89d1-c402ad6d90d8 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.021309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4fd31f5f-957b-472f-a587-0024351b72d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:28.024184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306a8b3-f8cb-41f4-aa91-fb6d81d6c4dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.024832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306a8b3-f8cb-41f4-aa91-fb6d81d6c4dc service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.025145 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306a8b3-f8cb-41f4-aa91-fb6d81d6c4dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.025420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306a8b3-f8cb-41f4-aa91-fb6d81d6c4dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.029763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1308/2647] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:28.034037 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31f5f-957b-472f-a587-0024351b72d2 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.034620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306a8b3-f8cb-41f4-aa91-fb6d81d6c4dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.036062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306a8b3-f8cb-41f4-aa91-fb6d81d6c4dc service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.042933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31f5f-957b-472f-a587-0024351b72d2 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.045031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1340/2648] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:37:28.047704 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd08437-116c-44df-8ab9-9645a513df3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.048384 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd08437-116c-44df-8ab9-9645a513df3f admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.048668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd08437-116c-44df-8ab9-9645a513df3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.049073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd08437-116c-44df-8ab9-9645a513df3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.049902 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd08437-116c-44df-8ab9-9645a513df3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:28.050153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd08437-116c-44df-8ab9-9645a513df3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.051783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd08437-116c-44df-8ab9-9645a513df3f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.063804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1309/2649] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:28.068802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306a8b3-f8cb-41f4-aa91-fb6d81d6c4dc service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.079450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306a8b3-f8cb-41f4-aa91-fb6d81d6c4dc service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.084448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047db64-27ee-4849-9d62-b04846819176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.085146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047db64-27ee-4849-9d62-b04846819176 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.085433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047db64-27ee-4849-9d62-b04846819176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.085749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047db64-27ee-4849-9d62-b04846819176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.086594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047db64-27ee-4849-9d62-b04846819176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.088935 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047db64-27ee-4849-9d62-b04846819176 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.119775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1341/2650] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:28 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:28.120978 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd08437-116c-44df-8ab9-9645a513df3f admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.132088 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd08437-116c-44df-8ab9-9645a513df3f admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.188773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989eef6-32fe-4a46-b43a-098f9539d766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.189180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989eef6-32fe-4a46-b43a-098f9539d766 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.189477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989eef6-32fe-4a46-b43a-098f9539d766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.189765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989eef6-32fe-4a46-b43a-098f9539d766 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.190453 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989eef6-32fe-4a46-b43a-098f9539d766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.192372 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989eef6-32fe-4a46-b43a-098f9539d766 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.211943 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1310/2651] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:28 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:28.216252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047db64-27ee-4849-9d62-b04846819176 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.228449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047db64-27ee-4849-9d62-b04846819176 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.233955 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd2e45-eb57-4f0d-9ec6-28c1ce8aa673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.234622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2e45-eb57-4f0d-9ec6-28c1ce8aa673 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.234906 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2e45-eb57-4f0d-9ec6-28c1ce8aa673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.235183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2e45-eb57-4f0d-9ec6-28c1ce8aa673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.235909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd2e45-eb57-4f0d-9ec6-28c1ce8aa673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.238244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd2e45-eb57-4f0d-9ec6-28c1ce8aa673 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.250863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1342/2652] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:28.255249 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989eef6-32fe-4a46-b43a-098f9539d766 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.265918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989eef6-32fe-4a46-b43a-098f9539d766 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.271013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef806c9-10c7-44d8-992a-f330511e064d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.271674 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef806c9-10c7-44d8-992a-f330511e064d admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.271960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef806c9-10c7-44d8-992a-f330511e064d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.272238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef806c9-10c7-44d8-992a-f330511e064d admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.273061 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef806c9-10c7-44d8-992a-f330511e064d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.274679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef806c9-10c7-44d8-992a-f330511e064d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.289701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1311/2653] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:28.291508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd2e45-eb57-4f0d-9ec6-28c1ce8aa673 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.302231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd2e45-eb57-4f0d-9ec6-28c1ce8aa673 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.307061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60c3ff-0f67-4919-bbb3-e5a3664b56d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.308019 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60c3ff-0f67-4919-bbb3-e5a3664b56d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.308396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60c3ff-0f67-4919-bbb3-e5a3664b56d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.308738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60c3ff-0f67-4919-bbb3-e5a3664b56d4 admin admin] PATH_INFO: `/v3/projects/0a3060b8fe0245e29e9fbfed08be713f/users/05970e54aa8041b8be303c40308116e2/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.324540 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60c3ff-0f67-4919-bbb3-e5a3664b56d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3060b8fe0245e29e9fbfed08be713f, user_id=05970e54aa8041b8be303c40308116e2, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.327239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60c3ff-0f67-4919-bbb3-e5a3664b56d4 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.342310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1343/2654] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:28 2022] PUT /identity/v3/projects/0a3060b8fe0245e29e9fbfed08be713f/users/05970e54aa8041b8be303c40308116e2/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:28.355257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949c1fa-067c-4fcc-8f40-7ee52e660592 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.355257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949c1fa-067c-4fcc-8f40-7ee52e660592 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.355257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949c1fa-067c-4fcc-8f40-7ee52e660592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.384774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-6949c1fa-067c-4fcc-8f40-7ee52e660592 None None] MFA Rules not processed for user `05970e54aa8041b8be303c40308116e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:28.386316 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6949c1fa-067c-4fcc-8f40-7ee52e660592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:28.462475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1312/2655] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:28.467146 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60c3ff-0f67-4919-bbb3-e5a3664b56d4 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.530253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60c3ff-0f67-4919-bbb3-e5a3664b56d4 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.535711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450fe6f-f03a-4520-a011-c467b4ad5e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.536367 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450fe6f-f03a-4520-a011-c467b4ad5e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.536656 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450fe6f-f03a-4520-a011-c467b4ad5e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.536965 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450fe6f-f03a-4520-a011-c467b4ad5e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.537662 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450fe6f-f03a-4520-a011-c467b4ad5e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.539568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450fe6f-f03a-4520-a011-c467b4ad5e70 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.563551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1344/2656] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:28 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:28.567716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949c1fa-067c-4fcc-8f40-7ee52e660592 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.578302 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949c1fa-067c-4fcc-8f40-7ee52e660592 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.608405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.637190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67505703-1875-4b18-8a66-62b696a03762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.637859 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67505703-1875-4b18-8a66-62b696a03762 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.638145 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67505703-1875-4b18-8a66-62b696a03762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.638424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67505703-1875-4b18-8a66-62b696a03762 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.639109 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67505703-1875-4b18-8a66-62b696a03762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.641086 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67505703-1875-4b18-8a66-62b696a03762 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.661396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1313/2657] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:28 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:28.665456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450fe6f-f03a-4520-a011-c467b4ad5e70 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.676294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450fe6f-f03a-4520-a011-c467b4ad5e70 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.680859 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f5af-6ffc-4591-ba0d-9a4b185be3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.681544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f5af-6ffc-4591-ba0d-9a4b185be3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.681830 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f5af-6ffc-4591-ba0d-9a4b185be3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.682104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f5af-6ffc-4591-ba0d-9a4b185be3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.682854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f5af-6ffc-4591-ba0d-9a4b185be3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.684635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f5af-6ffc-4591-ba0d-9a4b185be3b2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.695782 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1345/2658] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:28.699874 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67505703-1875-4b18-8a66-62b696a03762 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.709780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67505703-1875-4b18-8a66-62b696a03762 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.714607 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45b304-731d-4f51-bc3d-c733121dad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.715512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45b304-731d-4f51-bc3d-c733121dad9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.715794 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45b304-731d-4f51-bc3d-c733121dad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.716069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45b304-731d-4f51-bc3d-c733121dad9c admin admin] PATH_INFO: `/v3/projects/0bcb8ec450d44917858caf22e33afa4d/users/e4bf4cc7310e4f7a8306d31d8b7c4428/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.731080 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45b304-731d-4f51-bc3d-c733121dad9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcb8ec450d44917858caf22e33afa4d, user_id=e4bf4cc7310e4f7a8306d31d8b7c4428, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.733795 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45b304-731d-4f51-bc3d-c733121dad9c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.747217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1314/2659] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:28 2022] PUT /identity/v3/projects/0bcb8ec450d44917858caf22e33afa4d/users/e4bf4cc7310e4f7a8306d31d8b7c4428/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:28.751031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f5af-6ffc-4591-ba0d-9a4b185be3b2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.761066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f5af-6ffc-4591-ba0d-9a4b185be3b2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.812702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.820592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f9224-b83d-4f16-ab87-994540e48dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.821297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f9224-b83d-4f16-ab87-994540e48dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.821576 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f9224-b83d-4f16-ab87-994540e48dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.821851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f9224-b83d-4f16-ab87-994540e48dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.822574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f9224-b83d-4f16-ab87-994540e48dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.824210 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f9224-b83d-4f16-ab87-994540e48dbe admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.834190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1346/2660] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:28.838175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45b304-731d-4f51-bc3d-c733121dad9c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.849202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45b304-731d-4f51-bc3d-c733121dad9c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.854003 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98909709-d46b-4fe4-b6be-d3a98c1a27e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.854890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98909709-d46b-4fe4-b6be-d3a98c1a27e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.855176 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98909709-d46b-4fe4-b6be-d3a98c1a27e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.855454 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98909709-d46b-4fe4-b6be-d3a98c1a27e3 admin admin] PATH_INFO: `/v3/projects/0bcb8ec450d44917858caf22e33afa4d/users/e4bf4cc7310e4f7a8306d31d8b7c4428/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.857470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98909709-d46b-4fe4-b6be-d3a98c1a27e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcb8ec450d44917858caf22e33afa4d, user_id=e4bf4cc7310e4f7a8306d31d8b7c4428, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.860326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98909709-d46b-4fe4-b6be-d3a98c1a27e3 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.871764 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1315/2661] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:28 2022] PUT /identity/v3/projects/0bcb8ec450d44917858caf22e33afa4d/users/e4bf4cc7310e4f7a8306d31d8b7c4428/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:28.882368 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3c765-4fb8-4d93-9a27-11e02b03d885 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.882669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3c765-4fb8-4d93-9a27-11e02b03d885 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.882954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3c765-4fb8-4d93-9a27-11e02b03d885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.910671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d4c3c765-4fb8-4d93-9a27-11e02b03d885 None None] MFA Rules not processed for user `e4bf4cc7310e4f7a8306d31d8b7c4428`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:28.911889 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d4c3c765-4fb8-4d93-9a27-11e02b03d885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:28.979068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1347/2662] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:28.990844 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98909709-d46b-4fe4-b6be-d3a98c1a27e3 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.049094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98909709-d46b-4fe4-b6be-d3a98c1a27e3 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.053966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12cf83-6438-41ad-b64a-34eb66913434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.054709 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12cf83-6438-41ad-b64a-34eb66913434 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.055035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12cf83-6438-41ad-b64a-34eb66913434 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.055339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12cf83-6438-41ad-b64a-34eb66913434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.064693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12cf83-6438-41ad-b64a-34eb66913434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.066370 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12cf83-6438-41ad-b64a-34eb66913434 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.075849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1316/2663] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:29.104399 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3c765-4fb8-4d93-9a27-11e02b03d885 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.165456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3c765-4fb8-4d93-9a27-11e02b03d885 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.169985 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.170839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.171120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.171392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] PATH_INFO: `/v3/users/05970e54aa8041b8be303c40308116e2` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.172574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05970e54aa8041b8be303c40308116e2)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.174717 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.219643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05970e54aa8041b8be303c40308116e2'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.231909 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05970e54aa8041b8be303c40308116e2'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.239525 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05970e54aa8041b8be303c40308116e2'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.240244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05970e54aa8041b8be303c40308116e2'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.240959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05970e54aa8041b8be303c40308116e2'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.246317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.255722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1348/2664] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:29 2022] DELETE /identity/v3/users/05970e54aa8041b8be303c40308116e2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:29.260222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a8071-3222-4cc1-aea9-b02b765e5bed None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.260711 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a8071-3222-4cc1-aea9-b02b765e5bed None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.260827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a8071-3222-4cc1-aea9-b02b765e5bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.278431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d72a8071-3222-4cc1-aea9-b02b765e5bed None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:29.279626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d72a8071-3222-4cc1-aea9-b02b765e5bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:29.344020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1317/2665] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:29.354084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.355508 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:29.427898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54497436-2b7a-4cf8-8bdd-82b1e5a70039 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.431990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04143242-f267-4f21-84fd-d2730fd8c33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.432652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04143242-f267-4f21-84fd-d2730fd8c33d service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.432995 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04143242-f267-4f21-84fd-d2730fd8c33d service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.433279 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04143242-f267-4f21-84fd-d2730fd8c33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.443646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04143242-f267-4f21-84fd-d2730fd8c33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.445128 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04143242-f267-4f21-84fd-d2730fd8c33d service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.455167 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1349/2666] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:37:29.721615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a8071-3222-4cc1-aea9-b02b765e5bed None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.723211 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d72a8071-3222-4cc1-aea9-b02b765e5bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:29.739387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a8071-3222-4cc1-aea9-b02b765e5bed None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.744827 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.745716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.746001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.746277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] PATH_INFO: `/v3/projects/0a3060b8fe0245e29e9fbfed08be713f` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.747459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a3060b8fe0245e29e9fbfed08be713f)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.749441 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.773348 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.798598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a3060b8fe0245e29e9fbfed08be713f', 'request_id': 'req-1a5a1833-4947-4a5f-aadb-79a526d517cb'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.807909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a3060b8fe0245e29e9fbfed08be713f', 'request_id': 'req-1a5a1833-4947-4a5f-aadb-79a526d517cb'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.816956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a3060b8fe0245e29e9fbfed08be713f', 'request_id': 'req-1a5a1833-4947-4a5f-aadb-79a526d517cb'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.828750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] The token cache is being invalidate because project 0a3060b8fe0245e29e9fbfed08be713f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:29.829377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.829729 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.832035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1318/2667] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:29 2022] DELETE /identity/v3/projects/0a3060b8fe0245e29e9fbfed08be713f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:29.835907 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04143242-f267-4f21-84fd-d2730fd8c33d service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.846663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04143242-f267-4f21-84fd-d2730fd8c33d service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.911801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.912677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.912986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.913278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] PATH_INFO: `/v3/users/e4bf4cc7310e4f7a8306d31d8b7c4428` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.914460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4bf4cc7310e4f7a8306d31d8b7c4428)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.916380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.971876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4bf4cc7310e4f7a8306d31d8b7c4428'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.984291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4bf4cc7310e4f7a8306d31d8b7c4428'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.991732 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4bf4cc7310e4f7a8306d31d8b7c4428'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.992443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4bf4cc7310e4f7a8306d31d8b7c4428'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:29.993105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-b2dcf0ba-f9b3-40da-947e-193ed14eb3b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4bf4cc7310e4f7a8306d31d8b7c4428'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:30.006059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1350/2668] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:29 2022] DELETE /identity/v3/users/e4bf4cc7310e4f7a8306d31d8b7c4428 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:30.292077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.294045 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:30.310155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a1833-4947-4a5f-aadb-79a526d517cb admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.371873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.372694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.373017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.373297 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] PATH_INFO: `/v3/projects/0bcb8ec450d44917858caf22e33afa4d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.374533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bcb8ec450d44917858caf22e33afa4d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.376444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.426039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bcb8ec450d44917858caf22e33afa4d', 'request_id': 'req-f47c459a-66e7-4947-a828-6647618b1fcc'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:30.434502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bcb8ec450d44917858caf22e33afa4d', 'request_id': 'req-f47c459a-66e7-4947-a828-6647618b1fcc'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:30.442443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bcb8ec450d44917858caf22e33afa4d', 'request_id': 'req-f47c459a-66e7-4947-a828-6647618b1fcc'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:30.454592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] The token cache is being invalidate because project 0bcb8ec450d44917858caf22e33afa4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:30.455819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:30.456629 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f47c459a-66e7-4947-a828-6647618b1fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:30.462051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1319/2669] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:30 2022] DELETE /identity/v3/projects/0bcb8ec450d44917858caf22e33afa4d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:30.464473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778320f8-c651-4bcb-ac62-91e771699daa None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.465256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778320f8-c651-4bcb-ac62-91e771699daa None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.465256 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778320f8-c651-4bcb-ac62-91e771699daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.485897 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-778320f8-c651-4bcb-ac62-91e771699daa None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:30.488050 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-778320f8-c651-4bcb-ac62-91e771699daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:30.500730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:30.552585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1351/2670] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:30.560060 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47746c76-7b2b-47f7-8bca-1bd3f8331ff2 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.560354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47746c76-7b2b-47f7-8bca-1bd3f8331ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.560631 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47746c76-7b2b-47f7-8bca-1bd3f8331ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.579577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-47746c76-7b2b-47f7-8bca-1bd3f8331ff2 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:30.581455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-47746c76-7b2b-47f7-8bca-1bd3f8331ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:30.591468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1320/2671] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:30.595550 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778320f8-c651-4bcb-ac62-91e771699daa None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.605628 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778320f8-c651-4bcb-ac62-91e771699daa None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.610189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c092c5f-98f9-4582-8465-f2030c62159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.611057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c092c5f-98f9-4582-8465-f2030c62159a admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.611176 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c092c5f-98f9-4582-8465-f2030c62159a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.611416 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c092c5f-98f9-4582-8465-f2030c62159a admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.612179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c092c5f-98f9-4582-8465-f2030c62159a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:30.612481 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c092c5f-98f9-4582-8465-f2030c62159a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.614107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c092c5f-98f9-4582-8465-f2030c62159a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.625956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1352/2672] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:30.630278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47746c76-7b2b-47f7-8bca-1bd3f8331ff2 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.639588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47746c76-7b2b-47f7-8bca-1bd3f8331ff2 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.644909 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b83c50-f078-4cb9-9425-127cd8a14e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.645562 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b83c50-f078-4cb9-9425-127cd8a14e42 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.645852 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b83c50-f078-4cb9-9425-127cd8a14e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.646121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b83c50-f078-4cb9-9425-127cd8a14e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.646825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b83c50-f078-4cb9-9425-127cd8a14e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.648739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b83c50-f078-4cb9-9425-127cd8a14e42 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.675777 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1321/2673] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:30 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:30.680647 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c092c5f-98f9-4582-8465-f2030c62159a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.691815 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c092c5f-98f9-4582-8465-f2030c62159a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.752448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5c526-2a35-4753-ae1f-d28eb578912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.753222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5c526-2a35-4753-ae1f-d28eb578912d admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.753512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5c526-2a35-4753-ae1f-d28eb578912d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.753794 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5c526-2a35-4753-ae1f-d28eb578912d admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.754493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5c526-2a35-4753-ae1f-d28eb578912d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.756618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5c526-2a35-4753-ae1f-d28eb578912d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.791457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1353/2674] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:30 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:30.792658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b83c50-f078-4cb9-9425-127cd8a14e42 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.805018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b83c50-f078-4cb9-9425-127cd8a14e42 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.809826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fe18e-9975-4ab8-8a17-ac699911b34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.810563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fe18e-9975-4ab8-8a17-ac699911b34c admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.810916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fe18e-9975-4ab8-8a17-ac699911b34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.811293 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fe18e-9975-4ab8-8a17-ac699911b34c admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.812023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fe18e-9975-4ab8-8a17-ac699911b34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.813876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fe18e-9975-4ab8-8a17-ac699911b34c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.820033 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:30.828068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1322/2675] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:30.833165 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5c526-2a35-4753-ae1f-d28eb578912d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.844058 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5c526-2a35-4753-ae1f-d28eb578912d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.850224 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4a8d8-5f40-49bc-9619-07dee2239b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.850898 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4a8d8-5f40-49bc-9619-07dee2239b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.851181 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4a8d8-5f40-49bc-9619-07dee2239b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.851456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4a8d8-5f40-49bc-9619-07dee2239b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.852190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4a8d8-5f40-49bc-9619-07dee2239b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.853808 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4a8d8-5f40-49bc-9619-07dee2239b31 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.864961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1354/2676] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:30.869152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fe18e-9975-4ab8-8a17-ac699911b34c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.880379 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fe18e-9975-4ab8-8a17-ac699911b34c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.884918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a5139-25f8-4df9-a8ed-1a615276a223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.885786 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a5139-25f8-4df9-a8ed-1a615276a223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.886122 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a5139-25f8-4df9-a8ed-1a615276a223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.886432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a5139-25f8-4df9-a8ed-1a615276a223 admin admin] PATH_INFO: `/v3/projects/9456e3e2917e472181c820fb193f8639/users/5e0af5ddcbc94cbcb9b2840b987441ff/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.902610 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a5139-25f8-4df9-a8ed-1a615276a223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9456e3e2917e472181c820fb193f8639, user_id=5e0af5ddcbc94cbcb9b2840b987441ff, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.905317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a5139-25f8-4df9-a8ed-1a615276a223 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.918750 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1323/2677] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:30 2022] PUT /identity/v3/projects/9456e3e2917e472181c820fb193f8639/users/5e0af5ddcbc94cbcb9b2840b987441ff/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:30.930419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fa5fb-c1ce-4552-80ab-9c7b20d008e3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.930711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fa5fb-c1ce-4552-80ab-9c7b20d008e3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.930997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fa5fb-c1ce-4552-80ab-9c7b20d008e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.958357 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4a7fa5fb-c1ce-4552-80ab-9c7b20d008e3 None None] MFA Rules not processed for user `5e0af5ddcbc94cbcb9b2840b987441ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:30.959555 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4a7fa5fb-c1ce-4552-80ab-9c7b20d008e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:31.025578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1355/2678] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:31.030626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a5139-25f8-4df9-a8ed-1a615276a223 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.091434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a5139-25f8-4df9-a8ed-1a615276a223 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.096101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23d269-b04e-4069-9aae-ef5ee9c953d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.096748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23d269-b04e-4069-9aae-ef5ee9c953d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.097066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23d269-b04e-4069-9aae-ef5ee9c953d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.097346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23d269-b04e-4069-9aae-ef5ee9c953d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.098047 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23d269-b04e-4069-9aae-ef5ee9c953d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.099951 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23d269-b04e-4069-9aae-ef5ee9c953d5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.129184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1324/2679] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:31 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:31.133339 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fa5fb-c1ce-4552-80ab-9c7b20d008e3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.144542 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fa5fb-c1ce-4552-80ab-9c7b20d008e3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.153593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.204096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ac66b-64b1-4ef7-a775-c1cebc35b585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.204777 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ac66b-64b1-4ef7-a775-c1cebc35b585 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.205095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ac66b-64b1-4ef7-a775-c1cebc35b585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.205371 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ac66b-64b1-4ef7-a775-c1cebc35b585 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.206646 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ac66b-64b1-4ef7-a775-c1cebc35b585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.208575 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ac66b-64b1-4ef7-a775-c1cebc35b585 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.232614 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1356/2680] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:31 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:31.234111 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23d269-b04e-4069-9aae-ef5ee9c953d5 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.244681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23d269-b04e-4069-9aae-ef5ee9c953d5 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.250339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f69f19-ae93-4013-8e89-5bd46dd73df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.251004 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f69f19-ae93-4013-8e89-5bd46dd73df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.251299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f69f19-ae93-4013-8e89-5bd46dd73df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.251591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f69f19-ae93-4013-8e89-5bd46dd73df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.252891 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f69f19-ae93-4013-8e89-5bd46dd73df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.254513 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f69f19-ae93-4013-8e89-5bd46dd73df7 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.265434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1325/2681] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:31.270092 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ac66b-64b1-4ef7-a775-c1cebc35b585 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.280666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ac66b-64b1-4ef7-a775-c1cebc35b585 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.285618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e04f6a-b95d-4cc0-97b1-dec1da702e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.286599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e04f6a-b95d-4cc0-97b1-dec1da702e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.286882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e04f6a-b95d-4cc0-97b1-dec1da702e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.287157 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e04f6a-b95d-4cc0-97b1-dec1da702e29 admin admin] PATH_INFO: `/v3/projects/1683448b666845b6a9e4c64897a0df41/users/d4c9ff364d12406888990dc1b2c7fc7e/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.302422 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e04f6a-b95d-4cc0-97b1-dec1da702e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1683448b666845b6a9e4c64897a0df41, user_id=d4c9ff364d12406888990dc1b2c7fc7e, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.305115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e04f6a-b95d-4cc0-97b1-dec1da702e29 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.319896 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1357/2682] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:31 2022] PUT /identity/v3/projects/1683448b666845b6a9e4c64897a0df41/users/d4c9ff364d12406888990dc1b2c7fc7e/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:31.323964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f69f19-ae93-4013-8e89-5bd46dd73df7 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.335141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f69f19-ae93-4013-8e89-5bd46dd73df7 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.394656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b042a6-0480-4b14-8704-74856db22156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.395294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b042a6-0480-4b14-8704-74856db22156 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.395578 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b042a6-0480-4b14-8704-74856db22156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.395860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b042a6-0480-4b14-8704-74856db22156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.396627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b042a6-0480-4b14-8704-74856db22156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.398232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b042a6-0480-4b14-8704-74856db22156 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.412367 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1326/2683] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:31.413376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e04f6a-b95d-4cc0-97b1-dec1da702e29 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.423972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e04f6a-b95d-4cc0-97b1-dec1da702e29 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.429287 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23670c5c-794e-44d8-8d42-76acbf06b7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.430226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23670c5c-794e-44d8-8d42-76acbf06b7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.430645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23670c5c-794e-44d8-8d42-76acbf06b7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.430960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23670c5c-794e-44d8-8d42-76acbf06b7cb admin admin] PATH_INFO: `/v3/projects/1683448b666845b6a9e4c64897a0df41/users/d4c9ff364d12406888990dc1b2c7fc7e/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.433523 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23670c5c-794e-44d8-8d42-76acbf06b7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1683448b666845b6a9e4c64897a0df41, user_id=d4c9ff364d12406888990dc1b2c7fc7e, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.436455 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23670c5c-794e-44d8-8d42-76acbf06b7cb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.453265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1358/2684] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:31 2022] PUT /identity/v3/projects/1683448b666845b6a9e4c64897a0df41/users/d4c9ff364d12406888990dc1b2c7fc7e/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:31.461900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3ddd5-fb8b-463f-bfe7-500f06e96ae6 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.462233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3ddd5-fb8b-463f-bfe7-500f06e96ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.462657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3ddd5-fb8b-463f-bfe7-500f06e96ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.490315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e7a3ddd5-fb8b-463f-bfe7-500f06e96ae6 None None] MFA Rules not processed for user `d4c9ff364d12406888990dc1b2c7fc7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:31.491890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e7a3ddd5-fb8b-463f-bfe7-500f06e96ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:31.527840 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.563164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1327/2685] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:31.574554 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23670c5c-794e-44d8-8d42-76acbf06b7cb admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.576239 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-23670c5c-794e-44d8-8d42-76acbf06b7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:31.635034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23670c5c-794e-44d8-8d42-76acbf06b7cb admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.638672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24311f-a61d-4b99-9e09-497d3127992a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.639318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24311f-a61d-4b99-9e09-497d3127992a service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.639602 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24311f-a61d-4b99-9e09-497d3127992a service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.639884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24311f-a61d-4b99-9e09-497d3127992a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.649024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24311f-a61d-4b99-9e09-497d3127992a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.650396 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24311f-a61d-4b99-9e09-497d3127992a service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.660003 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1359/2686] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:32.213597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3ddd5-fb8b-463f-bfe7-500f06e96ae6 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.281303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3ddd5-fb8b-463f-bfe7-500f06e96ae6 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.286442 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.287564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.287870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.288148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] PATH_INFO: `/v3/users/5e0af5ddcbc94cbcb9b2840b987441ff` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.289375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0af5ddcbc94cbcb9b2840b987441ff)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.291307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.338970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0af5ddcbc94cbcb9b2840b987441ff'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.339693 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0af5ddcbc94cbcb9b2840b987441ff'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.347400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0af5ddcbc94cbcb9b2840b987441ff'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.348166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0af5ddcbc94cbcb9b2840b987441ff'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.360052 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0af5ddcbc94cbcb9b2840b987441ff'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.373408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1328/2687] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:32 2022] DELETE /identity/v3/users/5e0af5ddcbc94cbcb9b2840b987441ff => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:32.378348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6f299-9934-461d-9905-beb4b80ca1bc None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.378639 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6f299-9934-461d-9905-beb4b80ca1bc None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.378922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6f299-9934-461d-9905-beb4b80ca1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.397599 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-8cb6f299-9934-461d-9905-beb4b80ca1bc None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:32.398798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8cb6f299-9934-461d-9905-beb4b80ca1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:32.469385 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1360/2688] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:32.479705 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.481140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:32.538777 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670e02f-1152-4041-8051-5b9823c0d626 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.542574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460755ca-14ac-4ad7-959e-d32d952400c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.543313 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460755ca-14ac-4ad7-959e-d32d952400c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.543560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460755ca-14ac-4ad7-959e-d32d952400c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.543824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460755ca-14ac-4ad7-959e-d32d952400c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.552819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.558597 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460755ca-14ac-4ad7-959e-d32d952400c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.561339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460755ca-14ac-4ad7-959e-d32d952400c4 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.573006 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1329/2689] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:32.872002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6f299-9934-461d-9905-beb4b80ca1bc None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.873755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-8cb6f299-9934-461d-9905-beb4b80ca1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:32.887215 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6f299-9934-461d-9905-beb4b80ca1bc None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.894014 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.895144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.895428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.895702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] PATH_INFO: `/v3/projects/9456e3e2917e472181c820fb193f8639` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.896958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9456e3e2917e472181c820fb193f8639)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.898868 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.901870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.951247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9456e3e2917e472181c820fb193f8639', 'request_id': 'req-43423bb9-3016-467d-aa2d-df7f1bef980a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.957376 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9456e3e2917e472181c820fb193f8639', 'request_id': 'req-43423bb9-3016-467d-aa2d-df7f1bef980a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.962992 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9456e3e2917e472181c820fb193f8639', 'request_id': 'req-43423bb9-3016-467d-aa2d-df7f1bef980a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.975475 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] The token cache is being invalidate because project 9456e3e2917e472181c820fb193f8639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:32.975694 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.976312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.977881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1361/2690] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:32 2022] DELETE /identity/v3/projects/9456e3e2917e472181c820fb193f8639 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:32.981824 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460755ca-14ac-4ad7-959e-d32d952400c4 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.992353 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460755ca-14ac-4ad7-959e-d32d952400c4 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.048150 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.049034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.049839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.050169 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] PATH_INFO: `/v3/users/d4c9ff364d12406888990dc1b2c7fc7e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.051330 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c9ff364d12406888990dc1b2c7fc7e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.053276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.099830 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c9ff364d12406888990dc1b2c7fc7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.100492 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c9ff364d12406888990dc1b2c7fc7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.107505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c9ff364d12406888990dc1b2c7fc7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.108118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c9ff364d12406888990dc1b2c7fc7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.123464 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-4f232190-bc32-44ec-baab-390672a6b8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c9ff364d12406888990dc1b2c7fc7e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.136633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1330/2691] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:32 2022] DELETE /identity/v3/users/d4c9ff364d12406888990dc1b2c7fc7e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:33.413212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.414629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:33.435411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43423bb9-3016-467d-aa2d-df7f1bef980a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.499321 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.500191 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.500470 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.500744 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] PATH_INFO: `/v3/projects/1683448b666845b6a9e4c64897a0df41` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.502127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1683448b666845b6a9e4c64897a0df41)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.504094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.553583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1683448b666845b6a9e4c64897a0df41', 'request_id': 'req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.559877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1683448b666845b6a9e4c64897a0df41', 'request_id': 'req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.565489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1683448b666845b6a9e4c64897a0df41', 'request_id': 'req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.579300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] The token cache is being invalidate because project 1683448b666845b6a9e4c64897a0df41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:33.579593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.580213 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5a9311cf-e4c6-4b45-ab0c-a7b322fca4bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.582596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1362/2692] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:33 2022] DELETE /identity/v3/projects/1683448b666845b6a9e4c64897a0df41 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:33.587656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188ce16-df27-4857-ba01-69cdfb3b75ee None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.588113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188ce16-df27-4857-ba01-69cdfb3b75ee None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.588215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188ce16-df27-4857-ba01-69cdfb3b75ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.612095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-6188ce16-df27-4857-ba01-69cdfb3b75ee None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:33.613436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6188ce16-df27-4857-ba01-69cdfb3b75ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:33.674499 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1331/2693] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:33.682402 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff5bd7-5707-4636-91fe-84c87a97bcf3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.682762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff5bd7-5707-4636-91fe-84c87a97bcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.683112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff5bd7-5707-4636-91fe-84c87a97bcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.701645 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-e6ff5bd7-5707-4636-91fe-84c87a97bcf3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:33.702908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-e6ff5bd7-5707-4636-91fe-84c87a97bcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:33.711553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1363/2694] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:33.715716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188ce16-df27-4857-ba01-69cdfb3b75ee None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.726310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188ce16-df27-4857-ba01-69cdfb3b75ee None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.733062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779adc24-a8bc-41b0-8da9-189e5a785fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.733062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779adc24-a8bc-41b0-8da9-189e5a785fef admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.733062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779adc24-a8bc-41b0-8da9-189e5a785fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.733062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779adc24-a8bc-41b0-8da9-189e5a785fef admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.733373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779adc24-a8bc-41b0-8da9-189e5a785fef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:33.733373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779adc24-a8bc-41b0-8da9-189e5a785fef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.734765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779adc24-a8bc-41b0-8da9-189e5a785fef admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.744350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:33.748757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1332/2695] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:33.753459 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff5bd7-5707-4636-91fe-84c87a97bcf3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.762866 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff5bd7-5707-4636-91fe-84c87a97bcf3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.767643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadfb88-abf2-4290-9eb6-1ff4d8529029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.769163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadfb88-abf2-4290-9eb6-1ff4d8529029 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.769517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadfb88-abf2-4290-9eb6-1ff4d8529029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.769860 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadfb88-abf2-4290-9eb6-1ff4d8529029 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.770631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadfb88-abf2-4290-9eb6-1ff4d8529029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.772600 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadfb88-abf2-4290-9eb6-1ff4d8529029 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.786800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:33.802970 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1364/2696] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:33 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:33.808983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779adc24-a8bc-41b0-8da9-189e5a785fef admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.819743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779adc24-a8bc-41b0-8da9-189e5a785fef admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.891190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b601a-b766-409f-95c8-c6982ebdd569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.891867 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b601a-b766-409f-95c8-c6982ebdd569 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.892196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b601a-b766-409f-95c8-c6982ebdd569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.892498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b601a-b766-409f-95c8-c6982ebdd569 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.893702 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b601a-b766-409f-95c8-c6982ebdd569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.895792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b601a-b766-409f-95c8-c6982ebdd569 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.919940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1333/2697] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:33 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:33.927672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadfb88-abf2-4290-9eb6-1ff4d8529029 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.939902 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadfb88-abf2-4290-9eb6-1ff4d8529029 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.946293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f484dc-ad51-4bbb-b213-035acd6790b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.947195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f484dc-ad51-4bbb-b213-035acd6790b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.947568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f484dc-ad51-4bbb-b213-035acd6790b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.947932 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f484dc-ad51-4bbb-b213-035acd6790b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.948726 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f484dc-ad51-4bbb-b213-035acd6790b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.951637 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f484dc-ad51-4bbb-b213-035acd6790b5 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.969285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b601a-b766-409f-95c8-c6982ebdd569 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.970084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1365/2698] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:33.980555 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b601a-b766-409f-95c8-c6982ebdd569 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.986491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd786fe3-c7a6-4171-bcf9-472df2f94490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.987474 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd786fe3-c7a6-4171-bcf9-472df2f94490 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.987879 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd786fe3-c7a6-4171-bcf9-472df2f94490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.988341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd786fe3-c7a6-4171-bcf9-472df2f94490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.989317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd786fe3-c7a6-4171-bcf9-472df2f94490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.991568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd786fe3-c7a6-4171-bcf9-472df2f94490 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.008164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1334/2699] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:34.012936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f484dc-ad51-4bbb-b213-035acd6790b5 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.025954 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f484dc-ad51-4bbb-b213-035acd6790b5 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.031580 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b0cf6-ac8f-4ef4-9689-d98fdc1e4617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.032471 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b0cf6-ac8f-4ef4-9689-d98fdc1e4617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.032761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b0cf6-ac8f-4ef4-9689-d98fdc1e4617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.033070 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b0cf6-ac8f-4ef4-9689-d98fdc1e4617 admin admin] PATH_INFO: `/v3/projects/78dc702cae504b959cbf005b41b8e45e/users/675cb36a2dba4194aeb8b39ceafef255/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.048953 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b0cf6-ac8f-4ef4-9689-d98fdc1e4617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78dc702cae504b959cbf005b41b8e45e, user_id=675cb36a2dba4194aeb8b39ceafef255, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.051940 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b0cf6-ac8f-4ef4-9689-d98fdc1e4617 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.066956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1366/2700] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:34 2022] PUT /identity/v3/projects/78dc702cae504b959cbf005b41b8e45e/users/675cb36a2dba4194aeb8b39ceafef255/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:34.077916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfa29d-68cb-4951-97e6-060d98010720 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.078227 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfa29d-68cb-4951-97e6-060d98010720 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.078514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfa29d-68cb-4951-97e6-060d98010720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.120460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-63bfa29d-68cb-4951-97e6-060d98010720 None None] MFA Rules not processed for user `675cb36a2dba4194aeb8b39ceafef255`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:34.121726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-63bfa29d-68cb-4951-97e6-060d98010720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:34.233388 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1335/2701] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:34.294201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b0cf6-ac8f-4ef4-9689-d98fdc1e4617 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.296197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b23b0cf6-ac8f-4ef4-9689-d98fdc1e4617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:34.355528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b0cf6-ac8f-4ef4-9689-d98fdc1e4617 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.359115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523258e8-e8e6-4d03-8566-b4ab9345e2d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.359753 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523258e8-e8e6-4d03-8566-b4ab9345e2d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.360034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523258e8-e8e6-4d03-8566-b4ab9345e2d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.360311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523258e8-e8e6-4d03-8566-b4ab9345e2d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.369334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523258e8-e8e6-4d03-8566-b4ab9345e2d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.370755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523258e8-e8e6-4d03-8566-b4ab9345e2d3 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.380209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1367/2702] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:43.743554 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfa29d-68cb-4951-97e6-060d98010720 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:43.745024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-63bfa29d-68cb-4951-97e6-060d98010720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:43.811354 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfa29d-68cb-4951-97e6-060d98010720 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:43.816568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:43.817483 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:43.817945 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:43.818134 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] PATH_INFO: `/v3/users/f45b0b4d1d1e417abd99b2f18e18d6f5` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:43.819333 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f45b0b4d1d1e417abd99b2f18e18d6f5)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:43.821318 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:43.832517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:43.875448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f45b0b4d1d1e417abd99b2f18e18d6f5'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:43.876139 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f45b0b4d1d1e417abd99b2f18e18d6f5'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:43.884467 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f45b0b4d1d1e417abd99b2f18e18d6f5'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:43.885209 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f45b0b4d1d1e417abd99b2f18e18d6f5'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:43.898748 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f45b0b4d1d1e417abd99b2f18e18d6f5'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:43.912246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1336/2703] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:43 2022] DELETE /identity/v3/users/f45b0b4d1d1e417abd99b2f18e18d6f5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:43.916788 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da81b8-ca96-442b-979b-84332e6bd836 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:43.917296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da81b8-ca96-442b-979b-84332e6bd836 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:43.917433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da81b8-ca96-442b-979b-84332e6bd836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:43.934426 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-18da81b8-ca96-442b-979b-84332e6bd836 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:43.935631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-18da81b8-ca96-442b-979b-84332e6bd836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:43.993688 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1368/2704] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:44.003812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.005251 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:44.063290 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1527d2-973b-4f12-90a3-3907fe9ec1e0 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:44.068000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cd029-2960-4d50-a0b4-d21c13058497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:44.068655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cd029-2960-4d50-a0b4-d21c13058497 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.069001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cd029-2960-4d50-a0b4-d21c13058497 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.069249 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cd029-2960-4d50-a0b4-d21c13058497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.078460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cd029-2960-4d50-a0b4-d21c13058497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:44.079899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cd029-2960-4d50-a0b4-d21c13058497 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:44.090869 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1337/2705] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:44.366013 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da81b8-ca96-442b-979b-84332e6bd836 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.367473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-18da81b8-ca96-442b-979b-84332e6bd836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:44.381432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da81b8-ca96-442b-979b-84332e6bd836 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:44.385990 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:44.386997 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.387526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.387810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] PATH_INFO: `/v3/projects/9eb131e588084a328cfcbec2af85b84e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.389030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb131e588084a328cfcbec2af85b84e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:44.390980 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:44.441443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb131e588084a328cfcbec2af85b84e', 'request_id': 'req-2f7a01c1-be51-4fee-9f27-187c907bb4cd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.447643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb131e588084a328cfcbec2af85b84e', 'request_id': 'req-2f7a01c1-be51-4fee-9f27-187c907bb4cd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.449648 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:44.454096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb131e588084a328cfcbec2af85b84e', 'request_id': 'req-2f7a01c1-be51-4fee-9f27-187c907bb4cd'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.466200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cd029-2960-4d50-a0b4-d21c13058497 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.468048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-511cd029-2960-4d50-a0b4-d21c13058497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:44.470936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] The token cache is being invalidate because project 9eb131e588084a328cfcbec2af85b84e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:44.471234 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.472380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.474169 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1369/2706] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:44 2022] DELETE /identity/v3/projects/9eb131e588084a328cfcbec2af85b84e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:44.477745 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.479053 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:44.481951 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cd029-2960-4d50-a0b4-d21c13058497 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:44.552095 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a01c1-be51-4fee-9f27-187c907bb4cd admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:44.553819 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:44.555312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.555596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.555873 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] PATH_INFO: `/v3/users/e953b269bc00433381f03983cd575a80` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.557086 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e953b269bc00433381f03983cd575a80)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:44.557883 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:44.558768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.559049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.559520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] PATH_INFO: `/v3/users/de060964367d402ebd7cf4712c2fefbc` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.560382 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:44.561424 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=de060964367d402ebd7cf4712c2fefbc)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:44.564219 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:44.610996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:44.622405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de060964367d402ebd7cf4712c2fefbc'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.627762 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e953b269bc00433381f03983cd575a80'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.628431 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e953b269bc00433381f03983cd575a80'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.637433 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e953b269bc00433381f03983cd575a80'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.638118 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e953b269bc00433381f03983cd575a80'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.642991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de060964367d402ebd7cf4712c2fefbc'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.651222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de060964367d402ebd7cf4712c2fefbc'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.651994 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de060964367d402ebd7cf4712c2fefbc'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.652664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ddb4b437-4bc0-4fbd-9b1d-bf91ecaa2d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de060964367d402ebd7cf4712c2fefbc'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.653892 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e953b269bc00433381f03983cd575a80'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:44.670023 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1370/2707] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:44 2022] DELETE /identity/v3/users/de060964367d402ebd7cf4712c2fefbc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:44.671415 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1338/2708] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:44 2022] DELETE /identity/v3/users/e953b269bc00433381f03983cd575a80 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:44.678433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee087fab-ee79-4432-8a49-a95429cb8ed5 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.678790 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee087fab-ee79-4432-8a49-a95429cb8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.679130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee087fab-ee79-4432-8a49-a95429cb8ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.700655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ee087fab-ee79-4432-8a49-a95429cb8ed5 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:44.701998 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ee087fab-ee79-4432-8a49-a95429cb8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:44.777197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1371/2709] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:44.787778 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.789276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:44.861519 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42b2d4-06fc-48ac-9a7f-c79c3cbc8aad admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:44.867280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9935353-1f53-4776-8853-fd989a7e048e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:44.869366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935353-1f53-4776-8853-fd989a7e048e service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.869366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935353-1f53-4776-8853-fd989a7e048e service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.869366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935353-1f53-4776-8853-fd989a7e048e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.882666 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9935353-1f53-4776-8853-fd989a7e048e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:44.883568 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9935353-1f53-4776-8853-fd989a7e048e service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:44.895810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1339/2710] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:44.932414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee087fab-ee79-4432-8a49-a95429cb8ed5 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.933941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ee087fab-ee79-4432-8a49-a95429cb8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:44.947012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:44.954598 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee087fab-ee79-4432-8a49-a95429cb8ed5 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:44.962635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:44.964047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.964363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.964803 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] PATH_INFO: `/v3/users/675cb36a2dba4194aeb8b39ceafef255` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.966457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=675cb36a2dba4194aeb8b39ceafef255)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:44.969578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.009992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9935353-1f53-4776-8853-fd989a7e048e service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.012546 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e9935353-1f53-4776-8853-fd989a7e048e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.042252 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675cb36a2dba4194aeb8b39ceafef255'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.043043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9935353-1f53-4776-8853-fd989a7e048e service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.050776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.052152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.052436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.052701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] PATH_INFO: `/v3/projects/0a0675fefaad4b039b9cdd22c598c972` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.054174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0675fefaad4b039b9cdd22c598c972)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.056144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.057816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675cb36a2dba4194aeb8b39ceafef255'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.084829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675cb36a2dba4194aeb8b39ceafef255'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.086448 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675cb36a2dba4194aeb8b39ceafef255'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.089284 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f5b4742e-b303-4cbd-94a4-495d40a5f752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675cb36a2dba4194aeb8b39ceafef255'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.117916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0675fefaad4b039b9cdd22c598c972', 'request_id': 'req-73113f57-edab-489c-a2f2-b713ce14de24'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.121267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1372/2711] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:44 2022] DELETE /identity/v3/users/675cb36a2dba4194aeb8b39ceafef255 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:45.125465 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0675fefaad4b039b9cdd22c598c972', 'request_id': 'req-73113f57-edab-489c-a2f2-b713ce14de24'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.126112 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaabad-867d-4bf7-8a07-1acd6c49c9a3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.126723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaabad-867d-4bf7-8a07-1acd6c49c9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.127164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaabad-867d-4bf7-8a07-1acd6c49c9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.137980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0675fefaad4b039b9cdd22c598c972', 'request_id': 'req-73113f57-edab-489c-a2f2-b713ce14de24'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.156412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f5aaabad-867d-4bf7-8a07-1acd6c49c9a3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:45.157838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] The token cache is being invalidate because project 0a0675fefaad4b039b9cdd22c598c972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:45.157986 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f5aaabad-867d-4bf7-8a07-1acd6c49c9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.158135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.158825 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-73113f57-edab-489c-a2f2-b713ce14de24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.164613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1340/2712] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:45 2022] DELETE /identity/v3/projects/0a0675fefaad4b039b9cdd22c598c972 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:45.166032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ebc37-bfcf-4239-814c-66326fcadb42 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.166397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ebc37-bfcf-4239-814c-66326fcadb42 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.166692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ebc37-bfcf-4239-814c-66326fcadb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.187664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-e99ebc37-bfcf-4239-814c-66326fcadb42 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:45.189315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e99ebc37-bfcf-4239-814c-66326fcadb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.234843 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.257081 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1373/2713] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:45.259216 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaabad-867d-4bf7-8a07-1acd6c49c9a3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.260702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f5aaabad-867d-4bf7-8a07-1acd6c49c9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.275559 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaabad-867d-4bf7-8a07-1acd6c49c9a3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.284188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.285083 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.285314 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1341/2714] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:45.285417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.288908 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] PATH_INFO: `/v3/projects/e7dc97a4abe84effa3a24e933febb468` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.289076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ebc37-bfcf-4239-814c-66326fcadb42 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.293095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-e99ebc37-bfcf-4239-814c-66326fcadb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.293710 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7dc97a4abe84effa3a24e933febb468)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.298587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.362598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ebc37-bfcf-4239-814c-66326fcadb42 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.362887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7dc97a4abe84effa3a24e933febb468', 'request_id': 'req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.369115 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7dc97a4abe84effa3a24e933febb468', 'request_id': 'req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.375214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7dc97a4abe84effa3a24e933febb468', 'request_id': 'req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.386501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.391185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] The token cache is being invalidate because project e7dc97a4abe84effa3a24e933febb468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:45.391548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.393067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-e7503c17-c20f-4c0a-99b1-da9aa1ef2885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.395560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1374/2715] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:45 2022] DELETE /identity/v3/projects/e7dc97a4abe84effa3a24e933febb468 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:45.397008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bdd19-77f7-44c5-b09a-1dfcbe483366 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.398049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bdd19-77f7-44c5-b09a-1dfcbe483366 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.398587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bdd19-77f7-44c5-b09a-1dfcbe483366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.421417 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-391bdd19-77f7-44c5-b09a-1dfcbe483366 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:45.422723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-391bdd19-77f7-44c5-b09a-1dfcbe483366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.425326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa6ed3-011e-4a07-ba85-78f3a807e8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.425976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa6ed3-011e-4a07-ba85-78f3a807e8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.426262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa6ed3-011e-4a07-ba85-78f3a807e8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.426540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa6ed3-011e-4a07-ba85-78f3a807e8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.428440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-60aa6ed3-011e-4a07-ba85-78f3a807e8d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.485916 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1375/2716] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:45.490286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c9077-99e8-4fe9-8e5d-8af4bab43d34 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.490605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c9077-99e8-4fe9-8e5d-8af4bab43d34 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.490920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c9077-99e8-4fe9-8e5d-8af4bab43d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.494285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa6ed3-011e-4a07-ba85-78f3a807e8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.495739 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa6ed3-011e-4a07-ba85-78f3a807e8d6 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.506151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1342/2717] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:45.507026 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa6ed3-011e-4a07-ba85-78f3a807e8d6 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.509237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-345c9077-99e8-4fe9-8e5d-8af4bab43d34 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:45.510430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-345c9077-99e8-4fe9-8e5d-8af4bab43d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.516686 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa6ed3-011e-4a07-ba85-78f3a807e8d6 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.519414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1376/2718] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:45.522508 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456bef7-6d95-4179-a02e-9f06b72ed363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.523448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456bef7-6d95-4179-a02e-9f06b72ed363 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.523448 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456bef7-6d95-4179-a02e-9f06b72ed363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.523741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456bef7-6d95-4179-a02e-9f06b72ed363 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.524423 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456bef7-6d95-4179-a02e-9f06b72ed363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:45.524983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456bef7-6d95-4179-a02e-9f06b72ed363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.526671 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456bef7-6d95-4179-a02e-9f06b72ed363 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.527456 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42076a7-b830-481a-a648-8f86fbb15408 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.527754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42076a7-b830-481a-a648-8f86fbb15408 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.528289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42076a7-b830-481a-a648-8f86fbb15408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.540689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1343/2719] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:45.549436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456bef7-6d95-4179-a02e-9f06b72ed363 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.555388 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d42076a7-b830-481a-a648-8f86fbb15408 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:45.556660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d42076a7-b830-481a-a648-8f86fbb15408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.560012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456bef7-6d95-4179-a02e-9f06b72ed363 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.565792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263f755-248f-4253-866b-9abdac00aa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.566317 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263f755-248f-4253-866b-9abdac00aa2e admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.566591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263f755-248f-4253-866b-9abdac00aa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.566924 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263f755-248f-4253-866b-9abdac00aa2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.567622 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263f755-248f-4253-866b-9abdac00aa2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.568692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1377/2720] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:45.572685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263f755-248f-4253-866b-9abdac00aa2e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.574327 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42076a7-b830-481a-a648-8f86fbb15408 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.589936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42076a7-b830-481a-a648-8f86fbb15408 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.597514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975c5df-a4d8-4291-88cd-172f7d668cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.598548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975c5df-a4d8-4291-88cd-172f7d668cac admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.599028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975c5df-a4d8-4291-88cd-172f7d668cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.599307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975c5df-a4d8-4291-88cd-172f7d668cac admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.600056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975c5df-a4d8-4291-88cd-172f7d668cac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:45.600427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975c5df-a4d8-4291-88cd-172f7d668cac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.602068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975c5df-a4d8-4291-88cd-172f7d668cac admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.614650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1344/2721] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:45.615575 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263f755-248f-4253-866b-9abdac00aa2e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.618096 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1378/2722] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:45.622055 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975c5df-a4d8-4291-88cd-172f7d668cac admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.629106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.633435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975c5df-a4d8-4291-88cd-172f7d668cac admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.706486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263f755-248f-4253-866b-9abdac00aa2e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.708374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ff226-911e-418c-9900-fc75f90d2df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.709201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ff226-911e-418c-9900-fc75f90d2df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.709485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ff226-911e-418c-9900-fc75f90d2df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.709819 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ff226-911e-418c-9900-fc75f90d2df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.710519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ff226-911e-418c-9900-fc75f90d2df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.712801 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ff226-911e-418c-9900-fc75f90d2df3 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.713849 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30388d78-fbc8-45f5-b492-f261dc5b31e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.714600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30388d78-fbc8-45f5-b492-f261dc5b31e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.715034 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30388d78-fbc8-45f5-b492-f261dc5b31e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.715246 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30388d78-fbc8-45f5-b492-f261dc5b31e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.715954 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30388d78-fbc8-45f5-b492-f261dc5b31e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.718238 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30388d78-fbc8-45f5-b492-f261dc5b31e6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.741390 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1345/2723] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:45.742231 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1379/2724] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:45.745132 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30388d78-fbc8-45f5-b492-f261dc5b31e6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.747881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ff226-911e-418c-9900-fc75f90d2df3 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.756350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30388d78-fbc8-45f5-b492-f261dc5b31e6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.822411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ff226-911e-418c-9900-fc75f90d2df3 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.827183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4d88c-0796-4b00-847e-017de7c98bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.828050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4d88c-0796-4b00-847e-017de7c98bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.828409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4d88c-0796-4b00-847e-017de7c98bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.828857 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4d88c-0796-4b00-847e-017de7c98bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.829822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4d88c-0796-4b00-847e-017de7c98bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.832136 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4d88c-0796-4b00-847e-017de7c98bbf admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.832377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e7b25-65d9-4944-92fd-d4b2c74ac695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.833087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e7b25-65d9-4944-92fd-d4b2c74ac695 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.833499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e7b25-65d9-4944-92fd-d4b2c74ac695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.833901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e7b25-65d9-4944-92fd-d4b2c74ac695 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.834593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e7b25-65d9-4944-92fd-d4b2c74ac695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.838028 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e7b25-65d9-4944-92fd-d4b2c74ac695 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.843587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1346/2725] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:45.844517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4d88c-0796-4b00-847e-017de7c98bbf admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.847097 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-88c4d88c-0796-4b00-847e-017de7c98bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.852614 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.865296 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4d88c-0796-4b00-847e-017de7c98bbf admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.872725 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1380/2726] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:45.872918 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.873048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e7b25-65d9-4944-92fd-d4b2c74ac695 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.873143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.873143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.873143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] PATH_INFO: `/v3/projects/78dc702cae504b959cbf005b41b8e45e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.873143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=78dc702cae504b959cbf005b41b8e45e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.874996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.882832 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e7b25-65d9-4944-92fd-d4b2c74ac695 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.887489 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6abe7-535c-4b0d-b401-c01c341e43a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.888149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6abe7-535c-4b0d-b401-c01c341e43a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.888434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6abe7-535c-4b0d-b401-c01c341e43a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.888716 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6abe7-535c-4b0d-b401-c01c341e43a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.889622 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6abe7-535c-4b0d-b401-c01c341e43a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.891360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6abe7-535c-4b0d-b401-c01c341e43a2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.902623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1381/2727] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:45.903734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6abe7-535c-4b0d-b401-c01c341e43a2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.915752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6abe7-535c-4b0d-b401-c01c341e43a2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.928173 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78dc702cae504b959cbf005b41b8e45e', 'request_id': 'req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.933848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78dc702cae504b959cbf005b41b8e45e', 'request_id': 'req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.941725 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78dc702cae504b959cbf005b41b8e45e', 'request_id': 'req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.953993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] The token cache is being invalidate because project 78dc702cae504b959cbf005b41b8e45e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:45.954278 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.954929 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:45.957044 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1347/2728] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:45 2022] DELETE /identity/v3/projects/78dc702cae504b959cbf005b41b8e45e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:45.957842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.961566 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.975876 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02585398-77df-4045-9497-2da18532aaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.976553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02585398-77df-4045-9497-2da18532aaca admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.976837 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02585398-77df-4045-9497-2da18532aaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.977136 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02585398-77df-4045-9497-2da18532aaca admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.977857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02585398-77df-4045-9497-2da18532aaca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.979533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02585398-77df-4045-9497-2da18532aaca admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.981158 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.993188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1382/2729] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:45.993188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d629c4-6e4f-4e4a-b079-b11ac771f179 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.993188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d629c4-6e4f-4e4a-b079-b11ac771f179 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.993188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d629c4-6e4f-4e4a-b079-b11ac771f179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.011421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-99d629c4-6e4f-4e4a-b079-b11ac771f179 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:46.012630 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-99d629c4-6e4f-4e4a-b079-b11ac771f179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.021242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1383/2730] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:46.022036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d629c4-6e4f-4e4a-b079-b11ac771f179 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.023295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-99d629c4-6e4f-4e4a-b079-b11ac771f179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.035387 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d629c4-6e4f-4e4a-b079-b11ac771f179 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.038694 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5971c-7ced-4e7b-8f76-c24f623a0e5e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.039703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4baa144-8d0f-4d6a-8526-a1cb1f61dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.040354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baa144-8d0f-4d6a-8526-a1cb1f61dcfe admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.040638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baa144-8d0f-4d6a-8526-a1cb1f61dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.040945 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baa144-8d0f-4d6a-8526-a1cb1f61dcfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.041668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4baa144-8d0f-4d6a-8526-a1cb1f61dcfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.043258 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4baa144-8d0f-4d6a-8526-a1cb1f61dcfe admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.043577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42a7c6-d275-4c4c-9ad4-cada5318c8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.044461 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a7c6-d275-4c4c-9ad4-cada5318c8e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.044743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a7c6-d275-4c4c-9ad4-cada5318c8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.045055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a7c6-d275-4c4c-9ad4-cada5318c8e3 admin admin] PATH_INFO: `/v3/projects/62a1aaea32d740089d1d1f74e734c824/users/bcba529930bd42c691ced522fcbf496d/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.054214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1384/2731] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:46.058214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb58e9c-9407-4d08-a15d-dbc02c27b681 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.058503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb58e9c-9407-4d08-a15d-dbc02c27b681 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.058816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb58e9c-9407-4d08-a15d-dbc02c27b681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.059386 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42a7c6-d275-4c4c-9ad4-cada5318c8e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a1aaea32d740089d1d1f74e734c824, user_id=bcba529930bd42c691ced522fcbf496d, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.062143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42a7c6-d275-4c4c-9ad4-cada5318c8e3 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.076093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1348/2732] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:45 2022] PUT /identity/v3/projects/62a1aaea32d740089d1d1f74e734c824/users/bcba529930bd42c691ced522fcbf496d/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:46.077007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42a7c6-d275-4c4c-9ad4-cada5318c8e3 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.077773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-5bb58e9c-9407-4d08-a15d-dbc02c27b681 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:46.078963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5bb58e9c-9407-4d08-a15d-dbc02c27b681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.139363 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1385/2733] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:46.140572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69ce4f-6251-444b-8afa-bcfce8b0b8a8 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.140882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69ce4f-6251-444b-8afa-bcfce8b0b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.141178 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69ce4f-6251-444b-8afa-bcfce8b0b8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.147023 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42a7c6-d275-4c4c-9ad4-cada5318c8e3 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.152120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbfd40-6fa5-4f6c-b40a-1074e5e1ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.153334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbfd40-6fa5-4f6c-b40a-1074e5e1ca23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.153611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbfd40-6fa5-4f6c-b40a-1074e5e1ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.153885 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbfd40-6fa5-4f6c-b40a-1074e5e1ca23 admin admin] PATH_INFO: `/v3/projects/449514f02fbc485484110bde5a2018c0/users/01d7c2fa381541139ea7aebaf2ae688f/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.168253 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1d69ce4f-6251-444b-8afa-bcfce8b0b8a8 None None] MFA Rules not processed for user `bcba529930bd42c691ced522fcbf496d`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:46.168900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbfd40-6fa5-4f6c-b40a-1074e5e1ca23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=449514f02fbc485484110bde5a2018c0, user_id=01d7c2fa381541139ea7aebaf2ae688f, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.169487 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1d69ce4f-6251-444b-8afa-bcfce8b0b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.171529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbfd40-6fa5-4f6c-b40a-1074e5e1ca23 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.184782 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1349/2734] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:46 2022] PUT /identity/v3/projects/449514f02fbc485484110bde5a2018c0/users/01d7c2fa381541139ea7aebaf2ae688f/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:46.185649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbfd40-6fa5-4f6c-b40a-1074e5e1ca23 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.217076 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.237974 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1386/2735] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:46.239334 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa484ed-bf11-441a-8227-e547f70b8a2f None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.239623 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa484ed-bf11-441a-8227-e547f70b8a2f None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.239899 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa484ed-bf11-441a-8227-e547f70b8a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.251722 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbfd40-6fa5-4f6c-b40a-1074e5e1ca23 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.257105 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6fad-22e8-4bc4-b887-ed19f66524de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.257248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ffa484ed-bf11-441a-8227-e547f70b8a2f None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:46.257768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6fad-22e8-4bc4-b887-ed19f66524de admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.258051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6fad-22e8-4bc4-b887-ed19f66524de admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.258326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ffa484ed-bf11-441a-8227-e547f70b8a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.258444 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6fad-22e8-4bc4-b887-ed19f66524de admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.259045 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f6fad-22e8-4bc4-b887-ed19f66524de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:46.259357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f6fad-22e8-4bc4-b887-ed19f66524de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.260993 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f6fad-22e8-4bc4-b887-ed19f66524de admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.265895 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1387/2736] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:46.269882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa484ed-bf11-441a-8227-e547f70b8a2f None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.273371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1350/2737] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:46.277526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6fad-22e8-4bc4-b887-ed19f66524de admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.279718 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa484ed-bf11-441a-8227-e547f70b8a2f None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.285068 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890c054-8138-4ca4-9081-6e44c2ad56fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.285727 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890c054-8138-4ca4-9081-6e44c2ad56fb admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.286010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890c054-8138-4ca4-9081-6e44c2ad56fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.286288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890c054-8138-4ca4-9081-6e44c2ad56fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.286977 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890c054-8138-4ca4-9081-6e44c2ad56fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.287536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6fad-22e8-4bc4-b887-ed19f66524de admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.288960 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890c054-8138-4ca4-9081-6e44c2ad56fb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.292320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15abdd4-8761-478f-bf9f-134bc30ba86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.293025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15abdd4-8761-478f-bf9f-134bc30ba86a admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.293312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15abdd4-8761-478f-bf9f-134bc30ba86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.293588 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15abdd4-8761-478f-bf9f-134bc30ba86a admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.294299 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15abdd4-8761-478f-bf9f-134bc30ba86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.296199 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15abdd4-8761-478f-bf9f-134bc30ba86a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.310280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.316524 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1388/2738] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:46.317553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890c054-8138-4ca4-9081-6e44c2ad56fb admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.319140 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-a890c054-8138-4ca4-9081-6e44c2ad56fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.327817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1351/2739] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:46.328490 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15abdd4-8761-478f-bf9f-134bc30ba86a admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.380826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890c054-8138-4ca4-9081-6e44c2ad56fb admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.392087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15abdd4-8761-478f-bf9f-134bc30ba86a admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.396796 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5099-0254-425e-83e9-e0f24ea9cee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.397495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5099-0254-425e-83e9-e0f24ea9cee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.397771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5099-0254-425e-83e9-e0f24ea9cee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.398062 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5099-0254-425e-83e9-e0f24ea9cee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.398746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5099-0254-425e-83e9-e0f24ea9cee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.400670 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5099-0254-425e-83e9-e0f24ea9cee9 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.422688 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1352/2740] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:46.423500 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5099-0254-425e-83e9-e0f24ea9cee9 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.433379 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5099-0254-425e-83e9-e0f24ea9cee9 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.433761 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7368fec-a925-478d-9b0f-ab78e2552408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.434419 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7368fec-a925-478d-9b0f-ab78e2552408 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.434699 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7368fec-a925-478d-9b0f-ab78e2552408 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.434982 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7368fec-a925-478d-9b0f-ab78e2552408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.438049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7fd33-13cd-49f6-9995-e76b918a8aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.438731 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7fd33-13cd-49f6-9995-e76b918a8aea admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.439001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7fd33-13cd-49f6-9995-e76b918a8aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.439276 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7fd33-13cd-49f6-9995-e76b918a8aea admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.439956 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7fd33-13cd-49f6-9995-e76b918a8aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.442595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7fd33-13cd-49f6-9995-e76b918a8aea admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.445158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7368fec-a925-478d-9b0f-ab78e2552408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.446618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7368fec-a925-478d-9b0f-ab78e2552408 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.456934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1389/2741] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:46.458299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7368fec-a925-478d-9b0f-ab78e2552408 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.463473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1353/2742] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/users => generated 286 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:46.468329 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7fd33-13cd-49f6-9995-e76b918a8aea admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.473202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7368fec-a925-478d-9b0f-ab78e2552408 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.479246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343ad9b-20d0-4e48-b9f9-6a30468c72a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.480016 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343ad9b-20d0-4e48-b9f9-6a30468c72a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.480298 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343ad9b-20d0-4e48-b9f9-6a30468c72a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.480618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343ad9b-20d0-4e48-b9f9-6a30468c72a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.481237 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7fd33-13cd-49f6-9995-e76b918a8aea admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.481477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343ad9b-20d0-4e48-b9f9-6a30468c72a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.483294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343ad9b-20d0-4e48-b9f9-6a30468c72a4 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.486920 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1945c72-7c3c-4f09-a59a-4941f47d0dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.488208 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1945c72-7c3c-4f09-a59a-4941f47d0dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.488480 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1945c72-7c3c-4f09-a59a-4941f47d0dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.488820 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1945c72-7c3c-4f09-a59a-4941f47d0dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.490401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1945c72-7c3c-4f09-a59a-4941f47d0dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.491203 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1945c72-7c3c-4f09-a59a-4941f47d0dd5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.498374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1390/2743] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:46.499499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343ad9b-20d0-4e48-b9f9-6a30468c72a4 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.508718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1354/2744] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:46.509877 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1945c72-7c3c-4f09-a59a-4941f47d0dd5 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.515212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343ad9b-20d0-4e48-b9f9-6a30468c72a4 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.521184 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3164b-d82a-4e9c-a357-8047f0146de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.522079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3164b-d82a-4e9c-a357-8047f0146de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.522360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3164b-d82a-4e9c-a357-8047f0146de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.522636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3164b-d82a-4e9c-a357-8047f0146de2 admin admin] PATH_INFO: `/v3/projects/4ca60c0581454df6b9ee5b4b2bfa48b0/users/8102f0b678324dea8214ba771fa508ea/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.523043 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1945c72-7c3c-4f09-a59a-4941f47d0dd5 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.528501 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a35126-3226-4dee-b1cf-352d449c84e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.529379 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a35126-3226-4dee-b1cf-352d449c84e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.529643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a35126-3226-4dee-b1cf-352d449c84e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.529935 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a35126-3226-4dee-b1cf-352d449c84e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.530681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a35126-3226-4dee-b1cf-352d449c84e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.532446 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a35126-3226-4dee-b1cf-352d449c84e5 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.540082 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3164b-d82a-4e9c-a357-8047f0146de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca60c0581454df6b9ee5b4b2bfa48b0, user_id=8102f0b678324dea8214ba771fa508ea, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.542933 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3164b-d82a-4e9c-a357-8047f0146de2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.545356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1355/2745] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:46.550040 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a35126-3226-4dee-b1cf-352d449c84e5 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.557843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1391/2746] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:46 2022] PUT /identity/v3/projects/4ca60c0581454df6b9ee5b4b2bfa48b0/users/8102f0b678324dea8214ba771fa508ea/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:46.561693 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3164b-d82a-4e9c-a357-8047f0146de2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.562619 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a35126-3226-4dee-b1cf-352d449c84e5 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.628453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.648146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aade502-9af7-427e-b9b2-466d4798a0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.650168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aade502-9af7-427e-b9b2-466d4798a0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.650491 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aade502-9af7-427e-b9b2-466d4798a0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.650742 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aade502-9af7-427e-b9b2-466d4798a0a0 admin admin] PATH_INFO: `/v3/projects/4b7b56a844c04f0d97fc46c39c8159ce/users/a3cfce84e75b4f5c80476c3b05c21819/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.650882 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3164b-d82a-4e9c-a357-8047f0146de2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.657697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975c24d-3987-4350-9bc0-6fdf5f131b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.658374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975c24d-3987-4350-9bc0-6fdf5f131b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.658659 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975c24d-3987-4350-9bc0-6fdf5f131b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.658973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975c24d-3987-4350-9bc0-6fdf5f131b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.659780 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975c24d-3987-4350-9bc0-6fdf5f131b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.661942 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975c24d-3987-4350-9bc0-6fdf5f131b0d admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.669042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aade502-9af7-427e-b9b2-466d4798a0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7b56a844c04f0d97fc46c39c8159ce, user_id=a3cfce84e75b4f5c80476c3b05c21819, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.672221 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aade502-9af7-427e-b9b2-466d4798a0a0 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.676185 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1392/2747] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:46.678105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975c24d-3987-4350-9bc0-6fdf5f131b0d admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.688703 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1356/2748] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:46 2022] PUT /identity/v3/projects/4b7b56a844c04f0d97fc46c39c8159ce/users/a3cfce84e75b4f5c80476c3b05c21819/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:46.692971 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975c24d-3987-4350-9bc0-6fdf5f131b0d admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.695294 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761f88b-9766-4ea2-9888-a55e94437ecb None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.695747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761f88b-9766-4ea2-9888-a55e94437ecb None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.696190 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761f88b-9766-4ea2-9888-a55e94437ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.715726 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-a761f88b-9766-4ea2-9888-a55e94437ecb None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:46.717138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a761f88b-9766-4ea2-9888-a55e94437ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.743048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.764589 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e106b-e59f-4a83-9774-11ebe0929a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.766349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e106b-e59f-4a83-9774-11ebe0929a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.766636 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e106b-e59f-4a83-9774-11ebe0929a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.766912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e106b-e59f-4a83-9774-11ebe0929a5f admin admin] PATH_INFO: `/v3/projects/4ca60c0581454df6b9ee5b4b2bfa48b0/users/8102f0b678324dea8214ba771fa508ea/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.770152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e106b-e59f-4a83-9774-11ebe0929a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca60c0581454df6b9ee5b4b2bfa48b0, user_id=8102f0b678324dea8214ba771fa508ea, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.781188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e106b-e59f-4a83-9774-11ebe0929a5f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.799478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1393/2749] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:46 2022] PUT /identity/v3/projects/4ca60c0581454df6b9ee5b4b2bfa48b0/users/8102f0b678324dea8214ba771fa508ea/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:46.803449 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1357/2750] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:46.812039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659a35-f847-430f-a13b-0a9ccab5147b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.812404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659a35-f847-430f-a13b-0a9ccab5147b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.812755 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659a35-f847-430f-a13b-0a9ccab5147b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.817910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761f88b-9766-4ea2-9888-a55e94437ecb None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.819649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a761f88b-9766-4ea2-9888-a55e94437ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.845010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-86659a35-f847-430f-a13b-0a9ccab5147b None None] MFA Rules not processed for user `8102f0b678324dea8214ba771fa508ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:46.847008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-86659a35-f847-430f-a13b-0a9ccab5147b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.894966 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761f88b-9766-4ea2-9888-a55e94437ecb None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.932724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1394/2751] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:46.947873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659a35-f847-430f-a13b-0a9ccab5147b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.949485 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-86659a35-f847-430f-a13b-0a9ccab5147b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.955310 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59611e2-7c2d-4a6d-bdbd-b641a950ee71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.956459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59611e2-7c2d-4a6d-bdbd-b641a950ee71 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.956459 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59611e2-7c2d-4a6d-bdbd-b641a950ee71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.956791 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59611e2-7c2d-4a6d-bdbd-b641a950ee71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.017526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659a35-f847-430f-a13b-0a9ccab5147b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.021411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c67d-73c2-4bf0-99f3-95ed9537b54c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.022065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c67d-73c2-4bf0-99f3-95ed9537b54c service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.022348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c67d-73c2-4bf0-99f3-95ed9537b54c service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.022629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c67d-73c2-4bf0-99f3-95ed9537b54c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.028252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59611e2-7c2d-4a6d-bdbd-b641a950ee71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.029754 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59611e2-7c2d-4a6d-bdbd-b641a950ee71 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.033204 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8c67d-73c2-4bf0-99f3-95ed9537b54c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.035776 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8c67d-73c2-4bf0-99f3-95ed9537b54c service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.041432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1358/2752] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:47.041690 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:47.047505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1395/2753] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:47.118069 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59611e2-7c2d-4a6d-bdbd-b641a950ee71 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:47.128091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59611e2-7c2d-4a6d-bdbd-b641a950ee71 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.133460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.134503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.134781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.135571 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] PATH_INFO: `/v3/users/bcba529930bd42c691ced522fcbf496d` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.136946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcba529930bd42c691ced522fcbf496d)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.138990 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.171148 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:47.205345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcba529930bd42c691ced522fcbf496d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:47.208307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcba529930bd42c691ced522fcbf496d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:47.224153 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcba529930bd42c691ced522fcbf496d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:47.225095 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcba529930bd42c691ced522fcbf496d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:47.244678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcba529930bd42c691ced522fcbf496d'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:47.263147 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1359/2754] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:47 2022] DELETE /identity/v3/users/bcba529930bd42c691ced522fcbf496d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:47.269174 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78fc8b-a90f-4cd4-a407-051219e3eae1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.269548 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78fc8b-a90f-4cd4-a407-051219e3eae1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.269775 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78fc8b-a90f-4cd4-a407-051219e3eae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.322294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6f78fc8b-a90f-4cd4-a407-051219e3eae1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:47.324264 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6f78fc8b-a90f-4cd4-a407-051219e3eae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:47.423723 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1396/2755] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:47.448008 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:47.449606 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:47.537350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27702f04-95eb-402b-b118-00d67bb7ee86 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.541321 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8d9f8-4b11-43ae-bd75-74f4ef6e682c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.541976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8d9f8-4b11-43ae-bd75-74f4ef6e682c service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.542256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8d9f8-4b11-43ae-bd75-74f4ef6e682c service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.542530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8d9f8-4b11-43ae-bd75-74f4ef6e682c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.553443 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8d9f8-4b11-43ae-bd75-74f4ef6e682c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.555017 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8d9f8-4b11-43ae-bd75-74f4ef6e682c service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.566128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1360/2756] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:48.148884 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78fc8b-a90f-4cd4-a407-051219e3eae1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.150934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6f78fc8b-a90f-4cd4-a407-051219e3eae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:48.168236 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78fc8b-a90f-4cd4-a407-051219e3eae1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.176558 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.178217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.178821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.179214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] PATH_INFO: `/v3/projects/62a1aaea32d740089d1d1f74e734c824` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.198765 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a1aaea32d740089d1d1f74e734c824)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.202452 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.254089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40489c-c276-4ae3-ae12-7c5d889e6987 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.255421 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40489c-c276-4ae3-ae12-7c5d889e6987 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.255919 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40489c-c276-4ae3-ae12-7c5d889e6987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.276044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a1aaea32d740089d1d1f74e734c824', 'request_id': 'req-f448d776-fd4e-4335-8f32-220a7e4d3916'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.288342 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a1aaea32d740089d1d1f74e734c824', 'request_id': 'req-f448d776-fd4e-4335-8f32-220a7e4d3916'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.299890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a1aaea32d740089d1d1f74e734c824', 'request_id': 'req-f448d776-fd4e-4335-8f32-220a7e4d3916'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.300677 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-be40489c-c276-4ae3-ae12-7c5d889e6987 None None] MFA Rules not processed for user `a3cfce84e75b4f5c80476c3b05c21819`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:48.302798 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-be40489c-c276-4ae3-ae12-7c5d889e6987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:48.320247 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] The token cache is being invalidate because project 62a1aaea32d740089d1d1f74e734c824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:48.320804 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.322810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.325386 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1397/2757] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:48 2022] DELETE /identity/v3/projects/62a1aaea32d740089d1d1f74e734c824 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:48.329333 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.331624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:48.465801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1361/2758] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:48.476818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40489c-c276-4ae3-ae12-7c5d889e6987 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.478790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-be40489c-c276-4ae3-ae12-7c5d889e6987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:48.494088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448d776-fd4e-4335-8f32-220a7e4d3916 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.505294 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.506267 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.506626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.506979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] PATH_INFO: `/v3/users/8102f0b678324dea8214ba771fa508ea` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.513958 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8102f0b678324dea8214ba771fa508ea)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.516158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.559230 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:48.574502 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:48.603038 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40489c-c276-4ae3-ae12-7c5d889e6987 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.605218 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8102f0b678324dea8214ba771fa508ea'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.614740 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f884eee-ab87-4153-80fe-18c83de46e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.615734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f884eee-ab87-4153-80fe-18c83de46e86 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.616079 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f884eee-ab87-4153-80fe-18c83de46e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.616366 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f884eee-ab87-4153-80fe-18c83de46e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.622749 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8102f0b678324dea8214ba771fa508ea'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.627307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f884eee-ab87-4153-80fe-18c83de46e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.629822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f884eee-ab87-4153-80fe-18c83de46e86 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.632730 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8102f0b678324dea8214ba771fa508ea'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.633601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8102f0b678324dea8214ba771fa508ea'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.634350 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-6a2adbe5-c073-403b-83a7-8b108f3342d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8102f0b678324dea8214ba771fa508ea'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:48.640504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1362/2759] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:37:48.655246 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1398/2760] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:48 2022] DELETE /identity/v3/users/8102f0b678324dea8214ba771fa508ea => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:48.993060 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f884eee-ab87-4153-80fe-18c83de46e86 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.022050 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f884eee-ab87-4153-80fe-18c83de46e86 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.109187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.110380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.110672 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.111495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] PATH_INFO: `/v3/projects/4ca60c0581454df6b9ee5b4b2bfa48b0` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.112220 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca60c0581454df6b9ee5b4b2bfa48b0)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.114233 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.171025 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca60c0581454df6b9ee5b4b2bfa48b0', 'request_id': 'req-69d54f3f-4af3-42ed-9f00-1adb54d4a877'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:49.177406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca60c0581454df6b9ee5b4b2bfa48b0', 'request_id': 'req-69d54f3f-4af3-42ed-9f00-1adb54d4a877'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:49.186020 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca60c0581454df6b9ee5b4b2bfa48b0', 'request_id': 'req-69d54f3f-4af3-42ed-9f00-1adb54d4a877'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:49.203297 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] The token cache is being invalidate because project 4ca60c0581454df6b9ee5b4b2bfa48b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:49.203598 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:49.204272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-69d54f3f-4af3-42ed-9f00-1adb54d4a877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:49.207012 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1363/2761] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:48 2022] DELETE /identity/v3/projects/4ca60c0581454df6b9ee5b4b2bfa48b0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:49.211501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5379a6-0684-4674-be14-4d638bf7ec8b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.212036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5379a6-0684-4674-be14-4d638bf7ec8b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.212036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5379a6-0684-4674-be14-4d638bf7ec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.230473 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-9d5379a6-0684-4674-be14-4d638bf7ec8b None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:49.231702 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-9d5379a6-0684-4674-be14-4d638bf7ec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:49.298457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1399/2762] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:49.308013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec8758-4684-4acb-ba36-ba0253d77e00 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.308580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec8758-4684-4acb-ba36-ba0253d77e00 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.308580 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec8758-4684-4acb-ba36-ba0253d77e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.329178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-24ec8758-4684-4acb-ba36-ba0253d77e00 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:49.330561 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-24ec8758-4684-4acb-ba36-ba0253d77e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:49.341717 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1364/2763] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:49.346517 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5379a6-0684-4674-be14-4d638bf7ec8b None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.357433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5379a6-0684-4674-be14-4d638bf7ec8b None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.363641 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5810a-d932-4c2d-9347-fb6fd38bc714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.364552 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5810a-d932-4c2d-9347-fb6fd38bc714 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.364979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5810a-d932-4c2d-9347-fb6fd38bc714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.365382 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5810a-d932-4c2d-9347-fb6fd38bc714 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.366250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5810a-d932-4c2d-9347-fb6fd38bc714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:49.366666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5810a-d932-4c2d-9347-fb6fd38bc714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.368450 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5810a-d932-4c2d-9347-fb6fd38bc714 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.382905 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1400/2764] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:49.387657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec8758-4684-4acb-ba36-ba0253d77e00 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.399306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec8758-4684-4acb-ba36-ba0253d77e00 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.406341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a42483-752b-4044-b882-7372330fb4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.407282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a42483-752b-4044-b882-7372330fb4df admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.407628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a42483-752b-4044-b882-7372330fb4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.408014 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a42483-752b-4044-b882-7372330fb4df admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.409068 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a42483-752b-4044-b882-7372330fb4df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.411551 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a42483-752b-4044-b882-7372330fb4df admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.441144 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1365/2765] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:49 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:49.445919 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5810a-d932-4c2d-9347-fb6fd38bc714 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.459797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5810a-d932-4c2d-9347-fb6fd38bc714 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.561962 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44231d37-0674-4039-9f28-1d1687aa90ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.562993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44231d37-0674-4039-9f28-1d1687aa90ad admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.563463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44231d37-0674-4039-9f28-1d1687aa90ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.563934 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44231d37-0674-4039-9f28-1d1687aa90ad admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.567237 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44231d37-0674-4039-9f28-1d1687aa90ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.572959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44231d37-0674-4039-9f28-1d1687aa90ad admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.611088 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1401/2766] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:49 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:49.613922 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a42483-752b-4044-b882-7372330fb4df admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.631076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a42483-752b-4044-b882-7372330fb4df admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.644064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53600b9c-3226-44e2-a25f-c5de7e282f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.644970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53600b9c-3226-44e2-a25f-c5de7e282f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.645301 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53600b9c-3226-44e2-a25f-c5de7e282f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.645574 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53600b9c-3226-44e2-a25f-c5de7e282f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.646463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53600b9c-3226-44e2-a25f-c5de7e282f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.648604 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53600b9c-3226-44e2-a25f-c5de7e282f9c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.665798 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1366/2767] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:49.668979 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44231d37-0674-4039-9f28-1d1687aa90ad admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.681528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44231d37-0674-4039-9f28-1d1687aa90ad admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.689789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218085da-1e44-429d-9470-0fba3e9c2f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.691133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218085da-1e44-429d-9470-0fba3e9c2f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.691414 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218085da-1e44-429d-9470-0fba3e9c2f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.691689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218085da-1e44-429d-9470-0fba3e9c2f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.693036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218085da-1e44-429d-9470-0fba3e9c2f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.694087 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218085da-1e44-429d-9470-0fba3e9c2f4a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.706873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1402/2768] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:49.710797 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53600b9c-3226-44e2-a25f-c5de7e282f9c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.723007 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53600b9c-3226-44e2-a25f-c5de7e282f9c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.730626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc27f6-be11-4d92-81c7-d40797316cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.732085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc27f6-be11-4d92-81c7-d40797316cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.732596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc27f6-be11-4d92-81c7-d40797316cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.733085 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc27f6-be11-4d92-81c7-d40797316cb9 admin admin] PATH_INFO: `/v3/projects/f92de8c11cfc4565835f4b08f6ecacd3/users/499fd020cdb8450bb730a08949fe5f9d/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.752231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc27f6-be11-4d92-81c7-d40797316cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92de8c11cfc4565835f4b08f6ecacd3, user_id=499fd020cdb8450bb730a08949fe5f9d, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.757244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc27f6-be11-4d92-81c7-d40797316cb9 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.779980 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1367/2769] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:49 2022] PUT /identity/v3/projects/f92de8c11cfc4565835f4b08f6ecacd3/users/499fd020cdb8450bb730a08949fe5f9d/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:49.791163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ec422-c2f3-484e-af25-798ad59fb73b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.791457 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ec422-c2f3-484e-af25-798ad59fb73b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.791748 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ec422-c2f3-484e-af25-798ad59fb73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.817075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:49.831326 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-685ec422-c2f3-484e-af25-798ad59fb73b None None] MFA Rules not processed for user `499fd020cdb8450bb730a08949fe5f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:49.834209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-685ec422-c2f3-484e-af25-798ad59fb73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:49.955631 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1403/2770] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:49.960142 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc27f6-be11-4d92-81c7-d40797316cb9 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.972298 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:50.060510 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc27f6-be11-4d92-81c7-d40797316cb9 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.066577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43d6f1-f21a-4d6f-8d39-968da27ce5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.067394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43d6f1-f21a-4d6f-8d39-968da27ce5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.067744 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43d6f1-f21a-4d6f-8d39-968da27ce5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.068161 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43d6f1-f21a-4d6f-8d39-968da27ce5f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.069350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43d6f1-f21a-4d6f-8d39-968da27ce5f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.071594 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43d6f1-f21a-4d6f-8d39-968da27ce5f6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.104498 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156892a-fa72-4873-b93b-0260ed49ab0c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.105010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156892a-fa72-4873-b93b-0260ed49ab0c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.105468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156892a-fa72-4873-b93b-0260ed49ab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.106787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1368/2771] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:49 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:50.114200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43d6f1-f21a-4d6f-8d39-968da27ce5f6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.138854 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4156892a-fa72-4873-b93b-0260ed49ab0c None None] MFA Rules not processed for user `a3cfce84e75b4f5c80476c3b05c21819`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:50.141768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4156892a-fa72-4873-b93b-0260ed49ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:50.209357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43d6f1-f21a-4d6f-8d39-968da27ce5f6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.215451 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594c727-63e2-484f-9be0-ffd304cb63ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.216940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594c727-63e2-484f-9be0-ffd304cb63ba admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.216940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594c727-63e2-484f-9be0-ffd304cb63ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.216940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594c727-63e2-484f-9be0-ffd304cb63ba admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.217883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594c727-63e2-484f-9be0-ffd304cb63ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.220138 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594c727-63e2-484f-9be0-ffd304cb63ba admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.253401 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1404/2772] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:37:50.275378 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1369/2773] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:50 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:50.280158 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156892a-fa72-4873-b93b-0260ed49ab0c None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.292938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156892a-fa72-4873-b93b-0260ed49ab0c None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.300040 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952d33c-3de8-4904-9e80-000b37d2c7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.301295 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952d33c-3de8-4904-9e80-000b37d2c7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.301769 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952d33c-3de8-4904-9e80-000b37d2c7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.302202 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952d33c-3de8-4904-9e80-000b37d2c7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.303162 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952d33c-3de8-4904-9e80-000b37d2c7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.305002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952d33c-3de8-4904-9e80-000b37d2c7ce admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.319838 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1405/2774] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:50.324210 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594c727-63e2-484f-9be0-ffd304cb63ba admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.341115 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594c727-63e2-484f-9be0-ffd304cb63ba admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.347460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525bedc-94eb-452d-ae06-9d5669e491c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.348380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525bedc-94eb-452d-ae06-9d5669e491c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.348662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525bedc-94eb-452d-ae06-9d5669e491c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.348972 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525bedc-94eb-452d-ae06-9d5669e491c1 admin admin] PATH_INFO: `/v3/projects/f3e17ef7b8a24cc4971e6c04516c88c4/users/4bda0df81a174df0a6e6721a70adb62a/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.384741 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525bedc-94eb-452d-ae06-9d5669e491c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3e17ef7b8a24cc4971e6c04516c88c4, user_id=4bda0df81a174df0a6e6721a70adb62a, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.389205 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525bedc-94eb-452d-ae06-9d5669e491c1 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.414784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1370/2775] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:50 2022] PUT /identity/v3/projects/f3e17ef7b8a24cc4971e6c04516c88c4/users/4bda0df81a174df0a6e6721a70adb62a/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:50.419574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952d33c-3de8-4904-9e80-000b37d2c7ce admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.434041 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952d33c-3de8-4904-9e80-000b37d2c7ce admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.535821 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5543ad-58f2-4569-bf17-45b3a0de0e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.538214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5543ad-58f2-4569-bf17-45b3a0de0e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.538587 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5543ad-58f2-4569-bf17-45b3a0de0e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.538946 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5543ad-58f2-4569-bf17-45b3a0de0e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.541434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5543ad-58f2-4569-bf17-45b3a0de0e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.541632 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5543ad-58f2-4569-bf17-45b3a0de0e58 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.552840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1406/2776] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:50.557057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525bedc-94eb-452d-ae06-9d5669e491c1 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.574911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525bedc-94eb-452d-ae06-9d5669e491c1 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.582977 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c876e3-c8b0-4749-9544-0d446e06e15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.583884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c876e3-c8b0-4749-9544-0d446e06e15c admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.584180 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c876e3-c8b0-4749-9544-0d446e06e15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.584503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c876e3-c8b0-4749-9544-0d446e06e15c admin admin] PATH_INFO: `/v3/projects/f3e17ef7b8a24cc4971e6c04516c88c4/users/4bda0df81a174df0a6e6721a70adb62a/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.593487 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c876e3-c8b0-4749-9544-0d446e06e15c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3e17ef7b8a24cc4971e6c04516c88c4, user_id=4bda0df81a174df0a6e6721a70adb62a, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.598407 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c876e3-c8b0-4749-9544-0d446e06e15c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.617643 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1371/2777] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:50 2022] PUT /identity/v3/projects/f3e17ef7b8a24cc4971e6c04516c88c4/users/4bda0df81a174df0a6e6721a70adb62a/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:50.626652 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c607b-85d1-4b34-a56f-aea2aec3d269 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.626840 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c607b-85d1-4b34-a56f-aea2aec3d269 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.627449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c607b-85d1-4b34-a56f-aea2aec3d269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.669972 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-d22c607b-85d1-4b34-a56f-aea2aec3d269 None None] MFA Rules not processed for user `4bda0df81a174df0a6e6721a70adb62a`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:50.671564 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d22c607b-85d1-4b34-a56f-aea2aec3d269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:50.713855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:50.805164 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1407/2778] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:50.818599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c876e3-c8b0-4749-9544-0d446e06e15c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.823511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b2c876e3-c8b0-4749-9544-0d446e06e15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:50.927179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:50.955512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c876e3-c8b0-4749-9544-0d446e06e15c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.964013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f4ceb-8869-488b-b45f-726f8ceb95a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.965102 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f4ceb-8869-488b-b45f-726f8ceb95a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.965584 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f4ceb-8869-488b-b45f-726f8ceb95a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.965868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f4ceb-8869-488b-b45f-726f8ceb95a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.980356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f4ceb-8869-488b-b45f-726f8ceb95a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.982621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f4ceb-8869-488b-b45f-726f8ceb95a2 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:51.013152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1372/2779] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:51.182362 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c607b-85d1-4b34-a56f-aea2aec3d269 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:51.199196 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c607b-85d1-4b34-a56f-aea2aec3d269 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:51.206901 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75676e86-0905-49ac-a237-ab695a296203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:51.208624 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75676e86-0905-49ac-a237-ab695a296203 service cinder] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:51.209120 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75676e86-0905-49ac-a237-ab695a296203 service cinder] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:51.209584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75676e86-0905-49ac-a237-ab695a296203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:51.309153 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75676e86-0905-49ac-a237-ab695a296203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:51.313226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75676e86-0905-49ac-a237-ab695a296203 service cinder] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:51.330541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1408/2780] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:51.860507 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de50b1-7c75-47d7-9bd8-96427ca70378 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:51.861170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de50b1-7c75-47d7-9bd8-96427ca70378 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:51.861170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de50b1-7c75-47d7-9bd8-96427ca70378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:51.896992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-f1de50b1-7c75-47d7-9bd8-96427ca70378 None None] MFA Rules not processed for user `01d7c2fa381541139ea7aebaf2ae688f`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:51.898347 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-f1de50b1-7c75-47d7-9bd8-96427ca70378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:51.981986 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1373/2781] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:51.986189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75676e86-0905-49ac-a237-ab695a296203 service cinder] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:51.988293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-75676e86-0905-49ac-a237-ab695a296203 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:52.100249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75676e86-0905-49ac-a237-ab695a296203 service cinder] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.105492 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dfb24-33d7-418c-874d-bb2de7f98bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.106175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dfb24-33d7-418c-874d-bb2de7f98bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.106463 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dfb24-33d7-418c-874d-bb2de7f98bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.106743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dfb24-33d7-418c-874d-bb2de7f98bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.107443 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dfb24-33d7-418c-874d-bb2de7f98bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.109430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dfb24-33d7-418c-874d-bb2de7f98bf2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.138306 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1409/2782] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:51 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:52.142385 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de50b1-7c75-47d7-9bd8-96427ca70378 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.160665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de50b1-7c75-47d7-9bd8-96427ca70378 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.227228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac9e13-e284-41b4-826e-cfc9e1c4a38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.227952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9e13-e284-41b4-826e-cfc9e1c4a38f admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.228239 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9e13-e284-41b4-826e-cfc9e1c4a38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.228514 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9e13-e284-41b4-826e-cfc9e1c4a38f admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.229259 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac9e13-e284-41b4-826e-cfc9e1c4a38f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.231371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac9e13-e284-41b4-826e-cfc9e1c4a38f admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.255753 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1374/2783] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:52 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:52.262117 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dfb24-33d7-418c-874d-bb2de7f98bf2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.273410 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dfb24-33d7-418c-874d-bb2de7f98bf2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.278209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668c614-21a5-456b-a24c-c835cbbbf2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.279144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668c614-21a5-456b-a24c-c835cbbbf2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.279427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668c614-21a5-456b-a24c-c835cbbbf2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.279705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668c614-21a5-456b-a24c-c835cbbbf2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.280434 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668c614-21a5-456b-a24c-c835cbbbf2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.282061 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668c614-21a5-456b-a24c-c835cbbbf2b1 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.294031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1410/2784] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:52.297896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac9e13-e284-41b4-826e-cfc9e1c4a38f admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.308497 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac9e13-e284-41b4-826e-cfc9e1c4a38f admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.313168 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99b1bd-009f-4837-9417-f52b458141a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.314058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99b1bd-009f-4837-9417-f52b458141a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.314338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99b1bd-009f-4837-9417-f52b458141a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.314613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99b1bd-009f-4837-9417-f52b458141a8 admin admin] PATH_INFO: `/v3/projects/ff6357e23ce94690840a2e24ec0a6143/users/5e82d4a31aa64c17a51fd70ae80a8975/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.332695 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99b1bd-009f-4837-9417-f52b458141a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6357e23ce94690840a2e24ec0a6143, user_id=5e82d4a31aa64c17a51fd70ae80a8975, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.335992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99b1bd-009f-4837-9417-f52b458141a8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.351644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1375/2785] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:52 2022] PUT /identity/v3/projects/ff6357e23ce94690840a2e24ec0a6143/users/5e82d4a31aa64c17a51fd70ae80a8975/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:52.355797 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668c614-21a5-456b-a24c-c835cbbbf2b1 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.367650 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668c614-21a5-456b-a24c-c835cbbbf2b1 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.394469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:52.439404 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e1fb8-1a8c-44d5-9733-53ada9bfb071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.440118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e1fb8-1a8c-44d5-9733-53ada9bfb071 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.441248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e1fb8-1a8c-44d5-9733-53ada9bfb071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.441763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e1fb8-1a8c-44d5-9733-53ada9bfb071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.442553 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e1fb8-1a8c-44d5-9733-53ada9bfb071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.444171 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e1fb8-1a8c-44d5-9733-53ada9bfb071 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.455188 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1411/2786] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:52.459455 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99b1bd-009f-4837-9417-f52b458141a8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.482633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99b1bd-009f-4837-9417-f52b458141a8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.490845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afe272-a3ea-466e-a3d0-c9e3555f988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.491214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afe272-a3ea-466e-a3d0-c9e3555f988e admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.491311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afe272-a3ea-466e-a3d0-c9e3555f988e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.491655 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afe272-a3ea-466e-a3d0-c9e3555f988e admin admin] PATH_INFO: `/v3/projects/ff6357e23ce94690840a2e24ec0a6143/users/5e82d4a31aa64c17a51fd70ae80a8975/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.493967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afe272-a3ea-466e-a3d0-c9e3555f988e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6357e23ce94690840a2e24ec0a6143, user_id=5e82d4a31aa64c17a51fd70ae80a8975, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.504041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afe272-a3ea-466e-a3d0-c9e3555f988e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.542682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1376/2787] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:52 2022] PUT /identity/v3/projects/ff6357e23ce94690840a2e24ec0a6143/users/5e82d4a31aa64c17a51fd70ae80a8975/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:54.203878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e1fb8-1a8c-44d5-9733-53ada9bfb071 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:54.217982 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e1fb8-1a8c-44d5-9733-53ada9bfb071 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:54.301356 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:54.303711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.304291 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.304981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] PATH_INFO: `/v3/users/499fd020cdb8450bb730a08949fe5f9d` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.306810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=499fd020cdb8450bb730a08949fe5f9d)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:54.309322 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:54.386588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499fd020cdb8450bb730a08949fe5f9d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:54.405428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499fd020cdb8450bb730a08949fe5f9d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:54.416161 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499fd020cdb8450bb730a08949fe5f9d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:54.417182 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499fd020cdb8450bb730a08949fe5f9d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:54.417789 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499fd020cdb8450bb730a08949fe5f9d'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:54.434578 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1412/2788] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:54 2022] DELETE /identity/v3/users/499fd020cdb8450bb730a08949fe5f9d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:54.439014 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596aee-5510-471a-b419-efddd712d1c9 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.439533 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596aee-5510-471a-b419-efddd712d1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.439650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596aee-5510-471a-b419-efddd712d1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.459164 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-92596aee-5510-471a-b419-efddd712d1c9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:54.460389 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-92596aee-5510-471a-b419-efddd712d1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:54.501200 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:54.560177 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1377/2789] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:54.571924 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:54.573904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:54.662427 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:54.678021 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557949d5-da4c-46e4-8cc9-ae024952ca64 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:54.682570 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ac599-7b12-4920-a5a1-ef9c8d603a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:54.683223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ac599-7b12-4920-a5a1-ef9c8d603a5f service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.684194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ac599-7b12-4920-a5a1-ef9c8d603a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.684501 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ac599-7b12-4920-a5a1-ef9c8d603a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.698190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ac599-7b12-4920-a5a1-ef9c8d603a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:54.699939 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ac599-7b12-4920-a5a1-ef9c8d603a5f service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:54.716039 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1413/2790] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:55.128996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92596aee-5510-471a-b419-efddd712d1c9 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:55.130800 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-92596aee-5510-471a-b419-efddd712d1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:55.158558 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92596aee-5510-471a-b419-efddd712d1c9 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:55.164715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:55.166011 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:55.166528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:55.167503 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] PATH_INFO: `/v3/projects/f92de8c11cfc4565835f4b08f6ecacd3` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:55.174526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92de8c11cfc4565835f4b08f6ecacd3)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:55.177669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:55.252690 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92de8c11cfc4565835f4b08f6ecacd3', 'request_id': 'req-a9af42f8-5def-4907-8597-81e74ceafa2a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.261170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92de8c11cfc4565835f4b08f6ecacd3', 'request_id': 'req-a9af42f8-5def-4907-8597-81e74ceafa2a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.271835 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92de8c11cfc4565835f4b08f6ecacd3', 'request_id': 'req-a9af42f8-5def-4907-8597-81e74ceafa2a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.286595 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] The token cache is being invalidate because project f92de8c11cfc4565835f4b08f6ecacd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:55.286883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.287719 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.289758 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1378/2791] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:55 2022] DELETE /identity/v3/projects/f92de8c11cfc4565835f4b08f6ecacd3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:55.293635 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ac599-7b12-4920-a5a1-ef9c8d603a5f service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:55.303875 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ac599-7b12-4920-a5a1-ef9c8d603a5f service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:55.373063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:55.374062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:55.374440 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:55.374822 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] PATH_INFO: `/v3/users/4bda0df81a174df0a6e6721a70adb62a` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:55.376197 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bda0df81a174df0a6e6721a70adb62a)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:55.379065 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:55.469747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bda0df81a174df0a6e6721a70adb62a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.526606 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bda0df81a174df0a6e6721a70adb62a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.545872 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bda0df81a174df0a6e6721a70adb62a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.546920 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bda0df81a174df0a6e6721a70adb62a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.548034 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-fc5e7f2f-1031-4a42-b5db-2fd16cb6a447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bda0df81a174df0a6e6721a70adb62a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:55.601752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1414/2792] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:37:55 2022] DELETE /identity/v3/users/4bda0df81a174df0a6e6721a70adb62a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:56.045360 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:56.046903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:56.084779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af42f8-5def-4907-8597-81e74ceafa2a admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:56.240860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:56.241682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.241958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.242229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] PATH_INFO: `/v3/projects/f3e17ef7b8a24cc4971e6c04516c88c4` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.246140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3e17ef7b8a24cc4971e6c04516c88c4)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:56.248478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:56.335196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3e17ef7b8a24cc4971e6c04516c88c4', 'request_id': 'req-1c7df19a-70bd-4716-975c-76434ed80430'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:56.343409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:56.350970 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3e17ef7b8a24cc4971e6c04516c88c4', 'request_id': 'req-1c7df19a-70bd-4716-975c-76434ed80430'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:56.363256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3e17ef7b8a24cc4971e6c04516c88c4', 'request_id': 'req-1c7df19a-70bd-4716-975c-76434ed80430'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:56.392113 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] The token cache is being invalidate because project f3e17ef7b8a24cc4971e6c04516c88c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:56.392404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:56.393106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-1c7df19a-70bd-4716-975c-76434ed80430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:56.396621 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1379/2793] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:37:56 2022] DELETE /identity/v3/projects/f3e17ef7b8a24cc4971e6c04516c88c4 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:56.401345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c972b52-9448-43e4-af8d-07acdfca4bd3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.402192 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c972b52-9448-43e4-af8d-07acdfca4bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.402644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c972b52-9448-43e4-af8d-07acdfca4bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.440361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-6c972b52-9448-43e4-af8d-07acdfca4bd3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:56.450309 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-6c972b52-9448-43e4-af8d-07acdfca4bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:56.597496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1415/2794] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:56.609472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff4459-bbe2-4fe2-8f61-b8d3356a258d None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.609745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff4459-bbe2-4fe2-8f61-b8d3356a258d None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.609829 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff4459-bbe2-4fe2-8f61-b8d3356a258d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.639983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-96ff4459-bbe2-4fe2-8f61-b8d3356a258d None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:56.641262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-96ff4459-bbe2-4fe2-8f61-b8d3356a258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:56.664765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1380/2795] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:56.669274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c972b52-9448-43e4-af8d-07acdfca4bd3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:56.678152 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:56.688660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c972b52-9448-43e4-af8d-07acdfca4bd3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:56.709296 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8ea05-d98f-477a-81aa-807c6f756eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:56.710002 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8ea05-d98f-477a-81aa-807c6f756eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.710307 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8ea05-d98f-477a-81aa-807c6f756eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.710586 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8ea05-d98f-477a-81aa-807c6f756eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.718893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8ea05-d98f-477a-81aa-807c6f756eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:56.719206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8ea05-d98f-477a-81aa-807c6f756eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:56.720852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8ea05-d98f-477a-81aa-807c6f756eb2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:56.760054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff4459-bbe2-4fe2-8f61-b8d3356a258d None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:56.768428 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1416/2796] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:56.771143 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff4459-bbe2-4fe2-8f61-b8d3356a258d None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:56.776027 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c888050-dc78-4924-9cac-780e0072877e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:56.776938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c888050-dc78-4924-9cac-780e0072877e admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.777059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c888050-dc78-4924-9cac-780e0072877e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.777262 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c888050-dc78-4924-9cac-780e0072877e admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.777946 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c888050-dc78-4924-9cac-780e0072877e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:56.780921 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c888050-dc78-4924-9cac-780e0072877e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:56.812024 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1381/2797] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:37:56 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:56.816660 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8ea05-d98f-477a-81aa-807c6f756eb2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:56.827904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8ea05-d98f-477a-81aa-807c6f756eb2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:56.912638 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777def1-f541-4abf-8962-10e2501420c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:56.913365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777def1-f541-4abf-8962-10e2501420c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.913654 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777def1-f541-4abf-8962-10e2501420c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.913993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777def1-f541-4abf-8962-10e2501420c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.914703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777def1-f541-4abf-8962-10e2501420c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:56.916760 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777def1-f541-4abf-8962-10e2501420c0 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:56.941881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1417/2798] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:37:56 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:56.946064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c888050-dc78-4924-9cac-780e0072877e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:56.959266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c888050-dc78-4924-9cac-780e0072877e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:56.970254 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86367e-6f22-4305-91f0-44a9c5c52396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:56.970756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86367e-6f22-4305-91f0-44a9c5c52396 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.971036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86367e-6f22-4305-91f0-44a9c5c52396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.971306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86367e-6f22-4305-91f0-44a9c5c52396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.972083 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86367e-6f22-4305-91f0-44a9c5c52396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:56.973896 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86367e-6f22-4305-91f0-44a9c5c52396 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:56.985418 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1382/2799] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:56.990802 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777def1-f541-4abf-8962-10e2501420c0 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:57.007293 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777def1-f541-4abf-8962-10e2501420c0 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:57.014973 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0e18f-78ff-49a5-beb4-9599295425b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:57.017031 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0e18f-78ff-49a5-beb4-9599295425b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.017432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0e18f-78ff-49a5-beb4-9599295425b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.017818 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0e18f-78ff-49a5-beb4-9599295425b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.019285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0e18f-78ff-49a5-beb4-9599295425b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:57.022126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0e18f-78ff-49a5-beb4-9599295425b7 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:57.043833 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1418/2800] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:57.044547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86367e-6f22-4305-91f0-44a9c5c52396 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:57.061166 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86367e-6f22-4305-91f0-44a9c5c52396 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:57.068120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa45c67-e1ae-4956-93c5-cee5a4aacfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:57.069252 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa45c67-e1ae-4956-93c5-cee5a4aacfa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.069641 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa45c67-e1ae-4956-93c5-cee5a4aacfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.070042 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa45c67-e1ae-4956-93c5-cee5a4aacfa6 admin admin] PATH_INFO: `/v3/projects/420451a620164147a39d6b8f76690f19/users/35923ae2d29f41b5975fc7fcf9110e2c/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.093743 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa45c67-e1ae-4956-93c5-cee5a4aacfa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=420451a620164147a39d6b8f76690f19, user_id=35923ae2d29f41b5975fc7fcf9110e2c, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:57.098401 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa45c67-e1ae-4956-93c5-cee5a4aacfa6 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:57.130336 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1383/2801] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:37:57 2022] PUT /identity/v3/projects/420451a620164147a39d6b8f76690f19/users/35923ae2d29f41b5975fc7fcf9110e2c/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:57.130783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8af1df-1311-4045-b1a1-c3b5eebfc30b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.131072 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8af1df-1311-4045-b1a1-c3b5eebfc30b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.131351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8af1df-1311-4045-b1a1-c3b5eebfc30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.165071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ab8af1df-1311-4045-b1a1-c3b5eebfc30b None None] MFA Rules not processed for user `35923ae2d29f41b5975fc7fcf9110e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:57.166314 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ab8af1df-1311-4045-b1a1-c3b5eebfc30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:57.256351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1419/2802] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:57.267716 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa45c67-e1ae-4956-93c5-cee5a4aacfa6 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:57.270971 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-9fa45c67-e1ae-4956-93c5-cee5a4aacfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:57.405116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa45c67-e1ae-4956-93c5-cee5a4aacfa6 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:57.414089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9a196-4758-4c1f-9006-d5f6a8c9ea69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:57.414776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9a196-4758-4c1f-9006-d5f6a8c9ea69 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.415057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9a196-4758-4c1f-9006-d5f6a8c9ea69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.415334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9a196-4758-4c1f-9006-d5f6a8c9ea69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.435057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9a196-4758-4c1f-9006-d5f6a8c9ea69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:57.437958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9a196-4758-4c1f-9006-d5f6a8c9ea69 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:57.462165 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1384/2803] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:58.925658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679e77c-293d-4705-b017-2d8494818151 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.926514 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679e77c-293d-4705-b017-2d8494818151 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.926793 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679e77c-293d-4705-b017-2d8494818151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.954223 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-2679e77c-293d-4705-b017-2d8494818151 None None] MFA Rules not processed for user `5e82d4a31aa64c17a51fd70ae80a8975`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:58.955374 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2679e77c-293d-4705-b017-2d8494818151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:59.008679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:59.046079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1420/2804] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:59.059556 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9a196-4758-4c1f-9006-d5f6a8c9ea69 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:59.061196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-10f9a196-4758-4c1f-9006-d5f6a8c9ea69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:59.162473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9a196-4758-4c1f-9006-d5f6a8c9ea69 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.171191 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90005e7a-24a9-4a7c-a656-5bf90c2bfb17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.171992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005e7a-24a9-4a7c-a656-5bf90c2bfb17 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.172345 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005e7a-24a9-4a7c-a656-5bf90c2bfb17 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.172687 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005e7a-24a9-4a7c-a656-5bf90c2bfb17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.176880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-90005e7a-24a9-4a7c-a656-5bf90c2bfb17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:59.208965 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:59.286471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90005e7a-24a9-4a7c-a656-5bf90c2bfb17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.288526 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90005e7a-24a9-4a7c-a656-5bf90c2bfb17 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.299429 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1385/2805] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:37:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:38:00.996843 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679e77c-293d-4705-b017-2d8494818151 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:01.059458 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679e77c-293d-4705-b017-2d8494818151 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.063890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.064754 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.065063 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.065341 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] PATH_INFO: `/v3/users/35923ae2d29f41b5975fc7fcf9110e2c` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.067149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=35923ae2d29f41b5975fc7fcf9110e2c)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.069075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.116238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35923ae2d29f41b5975fc7fcf9110e2c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.129781 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35923ae2d29f41b5975fc7fcf9110e2c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.144831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35923ae2d29f41b5975fc7fcf9110e2c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.145664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35923ae2d29f41b5975fc7fcf9110e2c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.146113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35923ae2d29f41b5975fc7fcf9110e2c'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.161283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1421/2806] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:38:00 2022] DELETE /identity/v3/users/35923ae2d29f41b5975fc7fcf9110e2c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:01.165746 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc886cc-36be-42be-a006-7aecd258dff1 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.166759 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc886cc-36be-42be-a006-7aecd258dff1 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.167110 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc886cc-36be-42be-a006-7aecd258dff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.187836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-1fc886cc-36be-42be-a006-7aecd258dff1 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:01.189096 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1fc886cc-36be-42be-a006-7aecd258dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:01.273046 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1386/2807] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:01.274893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:01.276227 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:01.359111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3bad3-e5bb-4bf9-b672-540afd6503ad admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.364268 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8e99a-7067-42e0-9cff-ca01331de34c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.365020 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8e99a-7067-42e0-9cff-ca01331de34c service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.365625 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8e99a-7067-42e0-9cff-ca01331de34c service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.365762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8e99a-7067-42e0-9cff-ca01331de34c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.382328 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8e99a-7067-42e0-9cff-ca01331de34c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.383806 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8e99a-7067-42e0-9cff-ca01331de34c service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.397172 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1422/2808] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:38:01.766128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc886cc-36be-42be-a006-7aecd258dff1 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:01.768231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1fc886cc-36be-42be-a006-7aecd258dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:01.785995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc886cc-36be-42be-a006-7aecd258dff1 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.821486 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.822346 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.822683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.822967 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] PATH_INFO: `/v3/projects/420451a620164147a39d6b8f76690f19` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.825375 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=420451a620164147a39d6b8f76690f19)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.827381 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.896212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '420451a620164147a39d6b8f76690f19', 'request_id': 'req-f3b117fc-53ee-4f10-95c4-464bff876ac7'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.906521 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '420451a620164147a39d6b8f76690f19', 'request_id': 'req-f3b117fc-53ee-4f10-95c4-464bff876ac7'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.934831 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '420451a620164147a39d6b8f76690f19', 'request_id': 'req-f3b117fc-53ee-4f10-95c4-464bff876ac7'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.950343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] The token cache is being invalidate because project 420451a620164147a39d6b8f76690f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:38:01.950713 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.951505 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-f3b117fc-53ee-4f10-95c4-464bff876ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:01.953564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1387/2809] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:38:01 2022] DELETE /identity/v3/projects/420451a620164147a39d6b8f76690f19 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:38:01.958348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07eb7-27eb-4f50-a353-13cdebd830af None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.958669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07eb7-27eb-4f50-a353-13cdebd830af None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.958951 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07eb7-27eb-4f50-a353-13cdebd830af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.972809 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:01.979728 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-45c07eb7-27eb-4f50-a353-13cdebd830af None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:01.982405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-45c07eb7-27eb-4f50-a353-13cdebd830af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:02.051071 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1423/2810] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:02.058817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92e0a3-feb0-4653-8a3e-45e37a4dd9f9 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.059135 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92e0a3-feb0-4653-8a3e-45e37a4dd9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.059860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92e0a3-feb0-4653-8a3e-45e37a4dd9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.078327 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-bc92e0a3-feb0-4653-8a3e-45e37a4dd9f9 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:02.079466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-bc92e0a3-feb0-4653-8a3e-45e37a4dd9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:02.088494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1388/2811] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:02.092991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c07eb7-27eb-4f50-a353-13cdebd830af None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.104001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c07eb7-27eb-4f50-a353-13cdebd830af None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.108742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68554f-6ffa-4fce-b741-489e85cca0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.109472 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68554f-6ffa-4fce-b741-489e85cca0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.109760 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68554f-6ffa-4fce-b741-489e85cca0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.110043 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68554f-6ffa-4fce-b741-489e85cca0e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.110956 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68554f-6ffa-4fce-b741-489e85cca0e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:02.111289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68554f-6ffa-4fce-b741-489e85cca0e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.113311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68554f-6ffa-4fce-b741-489e85cca0e1 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.125449 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1424/2812] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:02.131495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92e0a3-feb0-4653-8a3e-45e37a4dd9f9 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.144337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92e0a3-feb0-4653-8a3e-45e37a4dd9f9 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.149472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a09e6-4212-47f8-8656-78cd963fbff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.150140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a09e6-4212-47f8-8656-78cd963fbff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.150424 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a09e6-4212-47f8-8656-78cd963fbff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.150701 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a09e6-4212-47f8-8656-78cd963fbff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.151396 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a09e6-4212-47f8-8656-78cd963fbff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.153470 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a09e6-4212-47f8-8656-78cd963fbff0 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.181536 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1389/2813] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:38:02 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:38:02.185640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68554f-6ffa-4fce-b741-489e85cca0e1 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.197200 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68554f-6ffa-4fce-b741-489e85cca0e1 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.257194 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44310-186a-42d7-acbf-01b4aa2b71f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.257874 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44310-186a-42d7-acbf-01b4aa2b71f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.258217 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44310-186a-42d7-acbf-01b4aa2b71f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.258502 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44310-186a-42d7-acbf-01b4aa2b71f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.259209 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44310-186a-42d7-acbf-01b4aa2b71f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.261190 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44310-186a-42d7-acbf-01b4aa2b71f0 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.283618 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1425/2814] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:38:02 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:38:02.288326 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a09e6-4212-47f8-8656-78cd963fbff0 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.308925 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a09e6-4212-47f8-8656-78cd963fbff0 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.310215 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4839e-9d5b-402b-8fb7-efa0692ed406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.311001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4839e-9d5b-402b-8fb7-efa0692ed406 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.311432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4839e-9d5b-402b-8fb7-efa0692ed406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.311765 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4839e-9d5b-402b-8fb7-efa0692ed406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.312560 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4839e-9d5b-402b-8fb7-efa0692ed406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.314412 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4839e-9d5b-402b-8fb7-efa0692ed406 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.329860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1390/2815] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:38:02.331878 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44310-186a-42d7-acbf-01b4aa2b71f0 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.342505 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44310-186a-42d7-acbf-01b4aa2b71f0 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.348290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb2e05-7509-4cd6-a8a5-d91967bc5bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.349130 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2e05-7509-4cd6-a8a5-d91967bc5bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.349423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2e05-7509-4cd6-a8a5-d91967bc5bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.349703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2e05-7509-4cd6-a8a5-d91967bc5bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.350442 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb2e05-7509-4cd6-a8a5-d91967bc5bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.352057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb2e05-7509-4cd6-a8a5-d91967bc5bcc admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.371392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1426/2816] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:38:02.372613 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4839e-9d5b-402b-8fb7-efa0692ed406 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.378182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:02.385627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4839e-9d5b-402b-8fb7-efa0692ed406 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.391438 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74b128-6e70-4319-b2ad-18bf8c98e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.392358 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74b128-6e70-4319-b2ad-18bf8c98e89e admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.392640 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74b128-6e70-4319-b2ad-18bf8c98e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.392944 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74b128-6e70-4319-b2ad-18bf8c98e89e admin admin] PATH_INFO: `/v3/projects/05e37c1d36204e1aab408ce6bfbe871a/users/88aedc3a8e2e4689a65fb176bfbf7762/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.410773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74b128-6e70-4319-b2ad-18bf8c98e89e admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e37c1d36204e1aab408ce6bfbe871a, user_id=88aedc3a8e2e4689a65fb176bfbf7762, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.413592 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74b128-6e70-4319-b2ad-18bf8c98e89e admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.428400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1391/2817] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:38:02 2022] PUT /identity/v3/projects/05e37c1d36204e1aab408ce6bfbe871a/users/88aedc3a8e2e4689a65fb176bfbf7762/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:02.439672 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b741a25-b8b6-4dfc-bcbb-462d4bb02905 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.440059 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b741a25-b8b6-4dfc-bcbb-462d4bb02905 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.440265 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b741a25-b8b6-4dfc-bcbb-462d4bb02905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.470519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0b741a25-b8b6-4dfc-bcbb-462d4bb02905 None None] MFA Rules not processed for user `88aedc3a8e2e4689a65fb176bfbf7762`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:02.472412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0b741a25-b8b6-4dfc-bcbb-462d4bb02905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:02.539857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1427/2818] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:02.551730 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74b128-6e70-4319-b2ad-18bf8c98e89e admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.553917 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-fd74b128-6e70-4319-b2ad-18bf8c98e89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:02.638235 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74b128-6e70-4319-b2ad-18bf8c98e89e admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.641976 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2fa06-48be-4427-a1af-c0cb46dfbaa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.642878 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2fa06-48be-4427-a1af-c0cb46dfbaa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.643553 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2fa06-48be-4427-a1af-c0cb46dfbaa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.644435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2fa06-48be-4427-a1af-c0cb46dfbaa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.656140 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2fa06-48be-4427-a1af-c0cb46dfbaa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.657814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2fa06-48be-4427-a1af-c0cb46dfbaa5 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.668214 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1392/2819] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:38:04.250106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04aa0d-2354-4f8f-8968-bc9474fd2ca0 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.250689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04aa0d-2354-4f8f-8968-bc9474fd2ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.250689 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04aa0d-2354-4f8f-8968-bc9474fd2ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.257010 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:04.271495 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-ab04aa0d-2354-4f8f-8968-bc9474fd2ca0 None None] MFA Rules not processed for user `88aedc3a8e2e4689a65fb176bfbf7762`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:04.272850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ab04aa0d-2354-4f8f-8968-bc9474fd2ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:04.285193 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1428/2820] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:06.474185 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2fa06-48be-4427-a1af-c0cb46dfbaa5 service cinder] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:06.475804 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-4fb2fa06-48be-4427-a1af-c0cb46dfbaa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:06.548594 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2fa06-48be-4427-a1af-c0cb46dfbaa5 service cinder] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:06.554355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777142-9690-4ab4-b29f-3ae8c898bfdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:06.555031 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777142-9690-4ab4-b29f-3ae8c898bfdd service placement] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:06.555309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777142-9690-4ab4-b29f-3ae8c898bfdd service placement] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:06.555540 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777142-9690-4ab4-b29f-3ae8c898bfdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:06.557380 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8a777142-9690-4ab4-b29f-3ae8c898bfdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:06.568810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777142-9690-4ab4-b29f-3ae8c898bfdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:06.570436 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777142-9690-4ab4-b29f-3ae8c898bfdd service placement] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:06.580530 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1393/2821] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:38:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:38:08.594893 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f53ee-c518-40b1-9879-71d2c49e8df8 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:08.595462 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f53ee-c518-40b1-9879-71d2c49e8df8 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:08.595563 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f53ee-c518-40b1-9879-71d2c49e8df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:08.626936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-715f53ee-c518-40b1-9879-71d2c49e8df8 None None] MFA Rules not processed for user `a3cfce84e75b4f5c80476c3b05c21819`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:08.628201 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-715f53ee-c518-40b1-9879-71d2c49e8df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:08.690154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1429/2822] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:38:16.575947 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:16.946642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6378e4-f425-41b1-a3d6-126c26e6e035 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.946989 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6378e4-f425-41b1-a3d6-126c26e6e035 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.947275 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6378e4-f425-41b1-a3d6-126c26e6e035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:16.965120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-0f6378e4-f425-41b1-a3d6-126c26e6e035 None None] MFA Rules not processed for user `01d7c2fa381541139ea7aebaf2ae688f`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:16.966430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0f6378e4-f425-41b1-a3d6-126c26e6e035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:17.025964 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1394/2823] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:18.691847 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=27 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:20.964398 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a416fa6-7523-4cf3-bc30-f4648c88698b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:20.967067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a416fa6-7523-4cf3-bc30-f4648c88698b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:20.967067 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a416fa6-7523-4cf3-bc30-f4648c88698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.984053 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-0a416fa6-7523-4cf3-bc30-f4648c88698b None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:20.985288 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-0a416fa6-7523-4cf3-bc30-f4648c88698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:21.083004 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1430/2824] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:38:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:21.138404 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6378e4-f425-41b1-a3d6-126c26e6e035 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:21.139858 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-0f6378e4-f425-41b1-a3d6-126c26e6e035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:21.219611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6378e4-f425-41b1-a3d6-126c26e6e035 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:21.226281 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c2310-5e1f-40ee-a1a0-03301b166251 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:21.227021 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c2310-5e1f-40ee-a1a0-03301b166251 service swift] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:21.227373 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c2310-5e1f-40ee-a1a0-03301b166251 service swift] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:21.227650 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c2310-5e1f-40ee-a1a0-03301b166251 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:21.237720 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c2310-5e1f-40ee-a1a0-03301b166251 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:21.240093 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c2310-5e1f-40ee-a1a0-03301b166251 service swift] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:21.248784 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1395/2825] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:38:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:38:21.804568 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d5d0-9dbc-408a-8c5b-e840ca80a055 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:21.804853 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d5d0-9dbc-408a-8c5b-e840ca80a055 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:21.805159 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d5d0-9dbc-408a-8c5b-e840ca80a055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:21.823263 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-2634d5d0-9dbc-408a-8c5b-e840ca80a055 None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:21.824442 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2634d5d0-9dbc-408a-8c5b-e840ca80a055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:21.831438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1431/2826] 10.222.0.30 () {58 vars in 967 bytes} [Mon May 23 15:38:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:21.843440 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c2310-5e1f-40ee-a1a0-03301b166251 service swift] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:21.854466 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c2310-5e1f-40ee-a1a0-03301b166251 service swift] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:21.860615 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a8c4f-9020-40dc-bd33-f00c1b3fe1c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:21.861306 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a8c4f-9020-40dc-bd33-f00c1b3fe1c7 service swift] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:21.861591 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a8c4f-9020-40dc-bd33-f00c1b3fe1c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:21.861868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a8c4f-9020-40dc-bd33-f00c1b3fe1c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:21.872768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a8c4f-9020-40dc-bd33-f00c1b3fe1c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:21.874638 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a8c4f-9020-40dc-bd33-f00c1b3fe1c7 service swift] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:21.884230 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1396/2827] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:38:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:38:22.088593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9c610-d7eb-44ef-b111-cfa17b871f1b None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.089118 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9c610-d7eb-44ef-b111-cfa17b871f1b None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.089214 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9c610-d7eb-44ef-b111-cfa17b871f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.107538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-bde9c610-d7eb-44ef-b111-cfa17b871f1b None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:22.108669 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-bde9c610-d7eb-44ef-b111-cfa17b871f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:22.116506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1432/2828] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:38:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:22.128212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a8c4f-9020-40dc-bd33-f00c1b3fe1c7 service swift] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:22.138912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a8c4f-9020-40dc-bd33-f00c1b3fe1c7 service swift] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:22.143910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab81666-e706-4ffd-908b-9ae484c6fb30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:22.144669 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab81666-e706-4ffd-908b-9ae484c6fb30 service swift] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.144996 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab81666-e706-4ffd-908b-9ae484c6fb30 service swift] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.145257 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab81666-e706-4ffd-908b-9ae484c6fb30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.153337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab81666-e706-4ffd-908b-9ae484c6fb30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:22.154933 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab81666-e706-4ffd-908b-9ae484c6fb30 service swift] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:22.163667 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1397/2829] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:38:22.494301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15811262-283e-4154-9931-f0372bc4b720 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.494596 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15811262-283e-4154-9931-f0372bc4b720 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.494877 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15811262-283e-4154-9931-f0372bc4b720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.513393 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-15811262-283e-4154-9931-f0372bc4b720 None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:22.514594 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-15811262-283e-4154-9931-f0372bc4b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:22.522180 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1433/2830] 10.222.0.30 () {58 vars in 967 bytes} [Mon May 23 15:38:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:22.538244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab81666-e706-4ffd-908b-9ae484c6fb30 service swift] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:22.549266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab81666-e706-4ffd-908b-9ae484c6fb30 service swift] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:22.556790 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82903c-dc02-4022-aa20-265288a2736e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:22.557801 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82903c-dc02-4022-aa20-265288a2736e service swift] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.558274 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82903c-dc02-4022-aa20-265288a2736e service swift] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.558745 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82903c-dc02-4022-aa20-265288a2736e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.572899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82903c-dc02-4022-aa20-265288a2736e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:22.574587 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82903c-dc02-4022-aa20-265288a2736e service swift] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:22.589066 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1398/2831] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:38:24.695518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15811262-283e-4154-9931-f0372bc4b720 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:24.767768 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15811262-283e-4154-9931-f0372bc4b720 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:24.773107 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:24.773984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:24.774423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:24.774706 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] PATH_INFO: `/v3/users/88aedc3a8e2e4689a65fb176bfbf7762` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:24.775950 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88aedc3a8e2e4689a65fb176bfbf7762)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:24.777913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:24.826317 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88aedc3a8e2e4689a65fb176bfbf7762'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:24.838560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88aedc3a8e2e4689a65fb176bfbf7762'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:24.845557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88aedc3a8e2e4689a65fb176bfbf7762'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:24.846151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88aedc3a8e2e4689a65fb176bfbf7762'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:24.846743 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88aedc3a8e2e4689a65fb176bfbf7762'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:24.859764 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1434/2832] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:38:24 2022] DELETE /identity/v3/users/88aedc3a8e2e4689a65fb176bfbf7762 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:24.864195 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e13f07-4d3a-461e-858f-4ae2d0c68bd6 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:24.864685 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e13f07-4d3a-461e-858f-4ae2d0c68bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:24.864780 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e13f07-4d3a-461e-858f-4ae2d0c68bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:24.882848 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-07e13f07-4d3a-461e-858f-4ae2d0c68bd6 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:24.884064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-07e13f07-4d3a-461e-858f-4ae2d0c68bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:24.933462 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=100 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:24.945516 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1399/2833] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:24.955325 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:24.958094 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:24.967255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:25.022175 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef108ca6-e650-41e5-b92a-bf9b39318d70 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.025933 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541f8aa-165f-491e-97bc-86fef3980231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.026584 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541f8aa-165f-491e-97bc-86fef3980231 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.026863 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541f8aa-165f-491e-97bc-86fef3980231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.027133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541f8aa-165f-491e-97bc-86fef3980231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.037154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541f8aa-165f-491e-97bc-86fef3980231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.038626 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541f8aa-165f-491e-97bc-86fef3980231 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:25.048551 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1435/2834] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:38:25.240133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e13f07-4d3a-461e-858f-4ae2d0c68bd6 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.241756 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-07e13f07-4d3a-461e-858f-4ae2d0c68bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:25.262981 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e13f07-4d3a-461e-858f-4ae2d0c68bd6 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.277120 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.278196 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.278547 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.278890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] PATH_INFO: `/v3/projects/05e37c1d36204e1aab408ce6bfbe871a` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.280156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e37c1d36204e1aab408ce6bfbe871a)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.282544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:25.332468 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e37c1d36204e1aab408ce6bfbe871a', 'request_id': 'req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:25.338661 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e37c1d36204e1aab408ce6bfbe871a', 'request_id': 'req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:25.346679 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e37c1d36204e1aab408ce6bfbe871a', 'request_id': 'req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:25.359240 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] The token cache is being invalidate because project 05e37c1d36204e1aab408ce6bfbe871a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:38:25.359532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:25.360172 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-eea3bacd-4bed-4bc8-942f-b7e5c707fbe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:25.361916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1400/2835] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:38:25 2022] DELETE /identity/v3/projects/05e37c1d36204e1aab408ce6bfbe871a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:25.367783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e5715-c0af-4060-a9ec-5b0526ea8d62 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.368075 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e5715-c0af-4060-a9ec-5b0526ea8d62 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.368361 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e5715-c0af-4060-a9ec-5b0526ea8d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.386787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-825e5715-c0af-4060-a9ec-5b0526ea8d62 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:25.388086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-825e5715-c0af-4060-a9ec-5b0526ea8d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:25.449913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1436/2836] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:25.457763 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215eb85-ca0d-466a-9998-cade2ef57c2e None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.458054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215eb85-ca0d-466a-9998-cade2ef57c2e None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.458341 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215eb85-ca0d-466a-9998-cade2ef57c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.483952 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d215eb85-ca0d-466a-9998-cade2ef57c2e None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:25.485779 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d215eb85-ca0d-466a-9998-cade2ef57c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:25.494839 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1401/2837] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:25.499608 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e5715-c0af-4060-a9ec-5b0526ea8d62 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.510967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e5715-c0af-4060-a9ec-5b0526ea8d62 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.516506 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f578d7-2003-4e40-a2de-9b127accaee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.517383 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f578d7-2003-4e40-a2de-9b127accaee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.517670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f578d7-2003-4e40-a2de-9b127accaee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.517949 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f578d7-2003-4e40-a2de-9b127accaee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.518686 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f578d7-2003-4e40-a2de-9b127accaee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:25.518993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f578d7-2003-4e40-a2de-9b127accaee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.521012 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f578d7-2003-4e40-a2de-9b127accaee9 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:25.536904 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1437/2838] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:25.541932 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215eb85-ca0d-466a-9998-cade2ef57c2e None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.556836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215eb85-ca0d-466a-9998-cade2ef57c2e None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.561810 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee79af-30ac-4ccb-bcee-56b56146658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.562498 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee79af-30ac-4ccb-bcee-56b56146658d admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.562783 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee79af-30ac-4ccb-bcee-56b56146658d admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.563060 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee79af-30ac-4ccb-bcee-56b56146658d admin admin] PATH_INFO: `/v3/projects` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.563749 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee79af-30ac-4ccb-bcee-56b56146658d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.565715 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee79af-30ac-4ccb-bcee-56b56146658d admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:25.597887 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1402/2839] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:38:25 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:38:25.602057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f578d7-2003-4e40-a2de-9b127accaee9 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.614435 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f578d7-2003-4e40-a2de-9b127accaee9 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.671477 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ac1cd-8fdf-4935-9d31-72787a003ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.672163 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ac1cd-8fdf-4935-9d31-72787a003ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.672445 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ac1cd-8fdf-4935-9d31-72787a003ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.672720 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ac1cd-8fdf-4935-9d31-72787a003ceb admin admin] PATH_INFO: `/v3/users` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.673469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ac1cd-8fdf-4935-9d31-72787a003ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.676049 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ac1cd-8fdf-4935-9d31-72787a003ceb admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:25.696226 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1438/2840] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:38:25 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:38:25.700430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee79af-30ac-4ccb-bcee-56b56146658d admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.710817 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee79af-30ac-4ccb-bcee-56b56146658d admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.715475 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efc6eb-2278-48b2-982b-72fe1b0d27c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.716124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efc6eb-2278-48b2-982b-72fe1b0d27c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.716406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efc6eb-2278-48b2-982b-72fe1b0d27c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.716681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efc6eb-2278-48b2-982b-72fe1b0d27c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.717434 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efc6eb-2278-48b2-982b-72fe1b0d27c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.719030 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efc6eb-2278-48b2-982b-72fe1b0d27c1 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:25.731035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1403/2841] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:38:25.735101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ac1cd-8fdf-4935-9d31-72787a003ceb admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.747135 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ac1cd-8fdf-4935-9d31-72787a003ceb admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.753024 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b6858-bdb9-40ae-9780-55e4004c29a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.753713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b6858-bdb9-40ae-9780-55e4004c29a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.753993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b6858-bdb9-40ae-9780-55e4004c29a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.754438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b6858-bdb9-40ae-9780-55e4004c29a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.755365 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b6858-bdb9-40ae-9780-55e4004c29a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.757857 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b6858-bdb9-40ae-9780-55e4004c29a2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:25.768286 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1439/2842] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:38:25.772853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efc6eb-2278-48b2-982b-72fe1b0d27c1 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.783968 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efc6eb-2278-48b2-982b-72fe1b0d27c1 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.789174 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506299b-9d81-438f-9d9d-e797f43a8e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.790057 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506299b-9d81-438f-9d9d-e797f43a8e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.790337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506299b-9d81-438f-9d9d-e797f43a8e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.790639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506299b-9d81-438f-9d9d-e797f43a8e03 admin admin] PATH_INFO: `/v3/projects/8fbd7396d6be4ec28f8df7c62b20d10b/users/434463d55cee4e25a2b0b4c70bded44e/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.805689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506299b-9d81-438f-9d9d-e797f43a8e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fbd7396d6be4ec28f8df7c62b20d10b, user_id=434463d55cee4e25a2b0b4c70bded44e, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.808352 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506299b-9d81-438f-9d9d-e797f43a8e03 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:25.821938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1404/2843] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:38:25 2022] PUT /identity/v3/projects/8fbd7396d6be4ec28f8df7c62b20d10b/users/434463d55cee4e25a2b0b4c70bded44e/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:25.827047 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17683c-2feb-4bdc-8882-b43a878e819c None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.827340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17683c-2feb-4bdc-8882-b43a878e819c None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.827644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17683c-2feb-4bdc-8882-b43a878e819c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.845138 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4c17683c-2feb-4bdc-8882-b43a878e819c None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:25.846340 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4c17683c-2feb-4bdc-8882-b43a878e819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:25.855179 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:25.909101 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1440/2844] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:25.919425 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506299b-9d81-438f-9d9d-e797f43a8e03 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.921212 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-c506299b-9d81-438f-9d9d-e797f43a8e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:25.981387 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506299b-9d81-438f-9d9d-e797f43a8e03 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.985125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca41f8-b9db-47c1-9362-d95b581e1c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.985771 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca41f8-b9db-47c1-9362-d95b581e1c69 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.986055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca41f8-b9db-47c1-9362-d95b581e1c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.986337 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca41f8-b9db-47c1-9362-d95b581e1c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.996736 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca41f8-b9db-47c1-9362-d95b581e1c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.998642 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca41f8-b9db-47c1-9362-d95b581e1c69 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.010229 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1405/2845] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:38:26.649539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936fa85-e750-4861-bdaa-eb131579fd79 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.649831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936fa85-e750-4861-bdaa-eb131579fd79 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.650111 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936fa85-e750-4861-bdaa-eb131579fd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.680303 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4936fa85-e750-4861-bdaa-eb131579fd79 None None] MFA Rules not processed for user `434463d55cee4e25a2b0b4c70bded44e`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:26.681590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4936fa85-e750-4861-bdaa-eb131579fd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:26.749138 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1441/2846] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:26.760910 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca41f8-b9db-47c1-9362-d95b581e1c69 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.762311 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-20ca41f8-b9db-47c1-9362-d95b581e1c69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:26.783495 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.822496 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca41f8-b9db-47c1-9362-d95b581e1c69 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.826213 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dfb42-7117-46f8-a4f5-cd92599cfe66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36daf80a4cbc4cfc8962c7924433d257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.826868 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dfb42-7117-46f8-a4f5-cd92599cfe66 service cinder] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.827152 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dfb42-7117-46f8-a4f5-cd92599cfe66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.827430 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dfb42-7117-46f8-a4f5-cd92599cfe66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.837572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580dfb42-7117-46f8-a4f5-cd92599cfe66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.839029 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580dfb42-7117-46f8-a4f5-cd92599cfe66 service cinder] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.848408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1406/2847] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:38:28.258036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecefa65-9421-48b0-90c9-c8d1efc43fd3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:28.258620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecefa65-9421-48b0-90c9-c8d1efc43fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:28.258620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecefa65-9421-48b0-90c9-c8d1efc43fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:28.276110 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-5ecefa65-9421-48b0-90c9-c8d1efc43fd3 None None] MFA Rules not processed for user `434463d55cee4e25a2b0b4c70bded44e`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:28.277370 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5ecefa65-9421-48b0-90c9-c8d1efc43fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:28.284773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1442/2848] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:38:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:31.612101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2a106-d262-4afe-acd3-b2bee4c32445 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:31.612611 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2a106-d262-4afe-acd3-b2bee4c32445 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:31.612727 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2a106-d262-4afe-acd3-b2bee4c32445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:31.630056 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-d1b2a106-d262-4afe-acd3-b2bee4c32445 None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:31.631277 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d1b2a106-d262-4afe-acd3-b2bee4c32445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:31.691035 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1407/2849] 10.222.0.30 () {58 vars in 1009 bytes} [Mon May 23 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:31.704238 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecefa65-9421-48b0-90c9-c8d1efc43fd3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:31.706325 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-5ecefa65-9421-48b0-90c9-c8d1efc43fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:31.792403 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecefa65-9421-48b0-90c9-c8d1efc43fd3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:31.799474 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed314493-77e5-4848-9561-552f34dc593f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:31.800280 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed314493-77e5-4848-9561-552f34dc593f service swift] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:31.800758 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed314493-77e5-4848-9561-552f34dc593f service swift] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:31.801557 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed314493-77e5-4848-9561-552f34dc593f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:31.811740 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed314493-77e5-4848-9561-552f34dc593f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:31.813380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed314493-77e5-4848-9561-552f34dc593f service swift] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:31.824460 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1443/2850] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:38:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:38:35.000303 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2a106-d262-4afe-acd3-b2bee4c32445 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:35.013680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2a106-d262-4afe-acd3-b2bee4c32445 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:35.018300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4dc73-9dd0-4b60-b322-c9947e14393d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:35.019183 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4dc73-9dd0-4b60-b322-c9947e14393d service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:35.020374 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4dc73-9dd0-4b60-b322-c9947e14393d service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:35.020853 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4dc73-9dd0-4b60-b322-c9947e14393d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:35.023000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-6bd4dc73-9dd0-4b60-b322-c9947e14393d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:35.105309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4dc73-9dd0-4b60-b322-c9947e14393d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:35.107228 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4dc73-9dd0-4b60-b322-c9947e14393d service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:35.130178 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1408/2851] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:38:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:38:41.820008 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=88 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:45.114633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=73 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:49.357541 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6196f8-f538-41bd-b150-136df24e26dd None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:49.358037 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6196f8-f538-41bd-b150-136df24e26dd None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:49.358144 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6196f8-f538-41bd-b150-136df24e26dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:49.377036 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-4a6196f8-f538-41bd-b150-136df24e26dd None None] MFA Rules not processed for user `0bb6b4503a4e43f480af8430e7f4fc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:49.378248 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4a6196f8-f538-41bd-b150-136df24e26dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:49.386573 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1444/2852] 10.222.0.30 () {58 vars in 967 bytes} [Mon May 23 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:38:49.399133 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4dc73-9dd0-4b60-b322-c9947e14393d service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:49.411907 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4dc73-9dd0-4b60-b322-c9947e14393d service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:49.417087 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79201cd1-eece-4e9d-acfa-72e8cbe97f78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6c489d950a4370bbf01866d2fc8fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:49.417735 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79201cd1-eece-4e9d-acfa-72e8cbe97f78 service swift] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:49.418013 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79201cd1-eece-4e9d-acfa-72e8cbe97f78 service swift] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:49.418285 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79201cd1-eece-4e9d-acfa-72e8cbe97f78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:49.426837 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79201cd1-eece-4e9d-acfa-72e8cbe97f78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:49.428414 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79201cd1-eece-4e9d-acfa-72e8cbe97f78 service swift] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:49.436331 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1409/2853] 10.222.0.30 () {60 vars in 1384 bytes} [Mon May 23 15:38:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:38:51.365301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6196f8-f538-41bd-b150-136df24e26dd None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:51.366778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4a6196f8-f538-41bd-b150-136df24e26dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:51.382397 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6196f8-f538-41bd-b150-136df24e26dd None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:51.387936 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:51.388831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:51.389147 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:51.389420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] PATH_INFO: `/v3/users/a3cfce84e75b4f5c80476c3b05c21819` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:51.390668 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3cfce84e75b4f5c80476c3b05c21819)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:51.392697 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:51.440283 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3cfce84e75b4f5c80476c3b05c21819'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.455030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3cfce84e75b4f5c80476c3b05c21819'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.464290 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3cfce84e75b4f5c80476c3b05c21819'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.465285 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3cfce84e75b4f5c80476c3b05c21819'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.466281 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-d4898dd7-0989-4140-850e-321d115a3d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3cfce84e75b4f5c80476c3b05c21819'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.482585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1445/2854] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:38:51 2022] DELETE /identity/v3/users/a3cfce84e75b4f5c80476c3b05c21819 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:51.665446 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79201cd1-eece-4e9d-acfa-72e8cbe97f78 service swift] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:51.676494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79201cd1-eece-4e9d-acfa-72e8cbe97f78 service swift] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:51.733048 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:51.733958 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:51.734256 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:51.734572 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] PATH_INFO: `/v3/projects/4b7b56a844c04f0d97fc46c39c8159ce` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:51.735860 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7b56a844c04f0d97fc46c39c8159ce)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:51.738049 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:51.784718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7b56a844c04f0d97fc46c39c8159ce', 'request_id': 'req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.791146 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7b56a844c04f0d97fc46c39c8159ce', 'request_id': 'req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.799036 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7b56a844c04f0d97fc46c39c8159ce', 'request_id': 'req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.810938 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] The token cache is being invalidate because project 4b7b56a844c04f0d97fc46c39c8159ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:38:51.811219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.812001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:38:51.813900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1410/2855] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:38:51 2022] DELETE /identity/v3/projects/4b7b56a844c04f0d97fc46c39c8159ce => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:39:01.478835 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:01.807786 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:35.966839 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc343af-266c-40ec-8f4f-413f382725a1 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:35.967381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc343af-266c-40ec-8f4f-413f382725a1 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:35.967381 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc343af-266c-40ec-8f4f-413f382725a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:35.986056 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-1fc343af-266c-40ec-8f4f-413f382725a1 None None] MFA Rules not processed for user `434463d55cee4e25a2b0b4c70bded44e`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:35.987274 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1fc343af-266c-40ec-8f4f-413f382725a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:36.040601 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1446/2856] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:39:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:46.033703 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=25 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:46.970528 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:46.972018 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:47.033432 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2ebc0-8ab5-4039-9f52-4ae7ee313b2a admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:47.038124 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:47.038723 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:47.038995 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:47.039273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:47.041094 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:47.112657 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:47.114217 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:47.125625 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1411/2857] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:57.120489 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=54 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:08.379566 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc343af-266c-40ec-8f4f-413f382725a1 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:08.381102 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1fc343af-266c-40ec-8f4f-413f382725a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:08.393887 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc343af-266c-40ec-8f4f-413f382725a1 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:08.397588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff991d-0a63-4603-b3ba-453080a026d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:08.398235 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff991d-0a63-4603-b3ba-453080a026d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:08.398520 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff991d-0a63-4603-b3ba-453080a026d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:08.398798 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff991d-0a63-4603-b3ba-453080a026d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:08.400536 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-88ff991d-0a63-4603-b3ba-453080a026d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:08.413993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff991d-0a63-4603-b3ba-453080a026d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:08.415415 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff991d-0a63-4603-b3ba-453080a026d8 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:08.426496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1447/2858] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:40:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:40:18.421677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=6 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:33.335691 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.337192 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:33.402582 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf25744-a6c6-4902-8e8c-6fe3daaae0d9 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:33.407656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:33.408517 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.408802 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.409106 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] PATH_INFO: `/v3/users/434463d55cee4e25a2b0b4c70bded44e` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.410272 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=434463d55cee4e25a2b0b4c70bded44e)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:33.412182 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:33.457059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434463d55cee4e25a2b0b4c70bded44e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.457639 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434463d55cee4e25a2b0b4c70bded44e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.464371 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434463d55cee4e25a2b0b4c70bded44e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.465055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434463d55cee4e25a2b0b4c70bded44e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.477411 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b8bc1a3b-bbcf-49d3-9598-f2c59032d1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434463d55cee4e25a2b0b4c70bded44e'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.490836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1412/2859] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:40:33 2022] DELETE /identity/v3/users/434463d55cee4e25a2b0b4c70bded44e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:40:33.658852 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff991d-0a63-4603-b3ba-453080a026d8 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.669663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff991d-0a63-4603-b3ba-453080a026d8 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:33.727308 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:33.728105 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.728384 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.728658 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] PATH_INFO: `/v3/projects/8fbd7396d6be4ec28f8df7c62b20d10b` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.729879 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fbd7396d6be4ec28f8df7c62b20d10b)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:33.732109 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:33.777189 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fbd7396d6be4ec28f8df7c62b20d10b', 'request_id': 'req-627739e8-8969-4043-a94a-2ff547052e04'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.783894 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fbd7396d6be4ec28f8df7c62b20d10b', 'request_id': 'req-627739e8-8969-4043-a94a-2ff547052e04'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.789574 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fbd7396d6be4ec28f8df7c62b20d10b', 'request_id': 'req-627739e8-8969-4043-a94a-2ff547052e04'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.801461 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] The token cache is being invalidate because project 8fbd7396d6be4ec28f8df7c62b20d10b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:40:33.801735 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.802348 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-627739e8-8969-4043-a94a-2ff547052e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:40:33.803870 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1448/2860] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:40:33 2022] DELETE /identity/v3/projects/8fbd7396d6be4ec28f8df7c62b20d10b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:40:33.809116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef3e6-22c2-4c16-b549-b2a3e10536de None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.809408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef3e6-22c2-4c16-b549-b2a3e10536de None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.809682 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef3e6-22c2-4c16-b549-b2a3e10536de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.828232 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-812ef3e6-22c2-4c16-b549-b2a3e10536de None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:33.829463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-812ef3e6-22c2-4c16-b549-b2a3e10536de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:33.892573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1413/2861] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:40:33.900750 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b0ae8-ae4e-4ecd-b9df-6c87d237e3a3 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.901069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b0ae8-ae4e-4ecd-b9df-6c87d237e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.901347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b0ae8-ae4e-4ecd-b9df-6c87d237e3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.920048 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-3e6b0ae8-ae4e-4ecd-b9df-6c87d237e3a3 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:33.921423 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-3e6b0ae8-ae4e-4ecd-b9df-6c87d237e3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:33.929763 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1449/2862] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:40:33.934627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ef3e6-22c2-4c16-b549-b2a3e10536de None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.944400 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ef3e6-22c2-4c16-b549-b2a3e10536de None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:33.949125 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34a4a-e149-468e-b87d-27e3e70c4d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:33.950397 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34a4a-e149-468e-b87d-27e3e70c4d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.950683 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34a4a-e149-468e-b87d-27e3e70c4d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.950959 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34a4a-e149-468e-b87d-27e3e70c4d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.951692 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34a4a-e149-468e-b87d-27e3e70c4d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:33.952001 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34a4a-e149-468e-b87d-27e3e70c4d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:33.953656 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34a4a-e149-468e-b87d-27e3e70c4d36 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:33.965300 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1414/2863] 10.222.0.30 () {58 vars in 1152 bytes} [Mon May 23 15:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:33.969588 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b0ae8-ae4e-4ecd-b9df-6c87d237e3a3 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.978993 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b0ae8-ae4e-4ecd-b9df-6c87d237e3a3 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:33.983828 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df0d1-6567-4645-914f-5d61be3af55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:33.984478 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df0d1-6567-4645-914f-5d61be3af55b admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.984762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df0d1-6567-4645-914f-5d61be3af55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.985069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df0d1-6567-4645-914f-5d61be3af55b admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.985800 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319df0d1-6567-4645-914f-5d61be3af55b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:33.987704 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319df0d1-6567-4645-914f-5d61be3af55b admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.013885 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1450/2864] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:40:33 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:40:34.017983 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34a4a-e149-468e-b87d-27e3e70c4d36 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:34.027826 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34a4a-e149-468e-b87d-27e3e70c4d36 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.031399 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:34.084718 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc9a39-58b1-4bb7-b08e-5fefddee91db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.085409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc9a39-58b1-4bb7-b08e-5fefddee91db admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.085689 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc9a39-58b1-4bb7-b08e-5fefddee91db admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.085960 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc9a39-58b1-4bb7-b08e-5fefddee91db admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.086633 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc9a39-58b1-4bb7-b08e-5fefddee91db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.088614 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc9a39-58b1-4bb7-b08e-5fefddee91db admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.107845 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1415/2865] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:40:34 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:40:34.111912 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df0d1-6567-4645-914f-5d61be3af55b admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:34.122590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df0d1-6567-4645-914f-5d61be3af55b admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.127086 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da6680-d920-4a50-b763-7941f6b838cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.127759 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da6680-d920-4a50-b763-7941f6b838cf admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.128060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da6680-d920-4a50-b763-7941f6b838cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.128352 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da6680-d920-4a50-b763-7941f6b838cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.129106 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da6680-d920-4a50-b763-7941f6b838cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.130677 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da6680-d920-4a50-b763-7941f6b838cf admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.141496 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1451/2866] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:40:34.145403 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc9a39-58b1-4bb7-b08e-5fefddee91db admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:34.155032 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc9a39-58b1-4bb7-b08e-5fefddee91db admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.160244 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d5c2-85a0-4373-b022-245823259fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.160998 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d5c2-85a0-4373-b022-245823259fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.161282 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d5c2-85a0-4373-b022-245823259fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.161557 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d5c2-85a0-4373-b022-245823259fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.162286 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598d5c2-85a0-4373-b022-245823259fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.164128 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598d5c2-85a0-4373-b022-245823259fb9 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.175472 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1416/2867] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:40:34.179531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da6680-d920-4a50-b763-7941f6b838cf admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:34.192360 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da6680-d920-4a50-b763-7941f6b838cf admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.197073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e65e64-4a57-4649-939e-7f64f85413d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.197983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e65e64-4a57-4649-939e-7f64f85413d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.198266 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e65e64-4a57-4649-939e-7f64f85413d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.198544 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e65e64-4a57-4649-939e-7f64f85413d2 admin admin] PATH_INFO: `/v3/projects/fe810dde94644f57b3b6262366c8f126/users/aec46cf4f87241a2a4581e34c154f5a8/roles/8de5789832e84f9388b1295d479fca6e` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.216126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e65e64-4a57-4649-939e-7f64f85413d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe810dde94644f57b3b6262366c8f126, user_id=aec46cf4f87241a2a4581e34c154f5a8, role_id=8de5789832e84f9388b1295d479fca6e)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.218851 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e65e64-4a57-4649-939e-7f64f85413d2 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.232529 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1452/2868] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:40:34 2022] PUT /identity/v3/projects/fe810dde94644f57b3b6262366c8f126/users/aec46cf4f87241a2a4581e34c154f5a8/roles/8de5789832e84f9388b1295d479fca6e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:40:34.237787 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735cda-31c5-451f-8f86-c2752013ce10 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.238077 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735cda-31c5-451f-8f86-c2752013ce10 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.238355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735cda-31c5-451f-8f86-c2752013ce10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.256408 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-21735cda-31c5-451f-8f86-c2752013ce10 None None] MFA Rules not processed for user `61ee072739a04036a34cf038cd44e02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:34.257644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-21735cda-31c5-451f-8f86-c2752013ce10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:34.320065 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1417/2869] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:40:34.329679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e65e64-4a57-4649-939e-7f64f85413d2 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:34.331420 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-57e65e64-4a57-4649-939e-7f64f85413d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:34.362433 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:34.390952 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e65e64-4a57-4649-939e-7f64f85413d2 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.394605 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeebbc9-9b9a-4021-a529-e06df295efb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.395249 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeebbc9-9b9a-4021-a529-e06df295efb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.395533 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeebbc9-9b9a-4021-a529-e06df295efb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.395816 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeebbc9-9b9a-4021-a529-e06df295efb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.405032 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeebbc9-9b9a-4021-a529-e06df295efb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.406453 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeebbc9-9b9a-4021-a529-e06df295efb2 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.415987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1453/2870] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:40:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:40:38.384912 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f795d-f425-430d-ba3a-2fb264952195 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.385403 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f795d-f425-430d-ba3a-2fb264952195 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.385519 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f795d-f425-430d-ba3a-2fb264952195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.421838 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-710f795d-f425-430d-ba3a-2fb264952195 None None] MFA Rules not processed for user `aec46cf4f87241a2a4581e34c154f5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:38.423051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-710f795d-f425-430d-ba3a-2fb264952195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:38.490883 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1418/2871] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:40:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:40:38.495351 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeebbc9-9b9a-4021-a529-e06df295efb2 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.505805 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeebbc9-9b9a-4021-a529-e06df295efb2 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.510366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac7f31-3b61-47ac-98a0-bb4fb530885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.511022 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac7f31-3b61-47ac-98a0-bb4fb530885e admin admin] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.511312 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac7f31-3b61-47ac-98a0-bb4fb530885e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.511592 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac7f31-3b61-47ac-98a0-bb4fb530885e admin admin] PATH_INFO: `/v3/projects` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.513069 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac7f31-3b61-47ac-98a0-bb4fb530885e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.514996 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac7f31-3b61-47ac-98a0-bb4fb530885e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.542310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1454/2872] 10.222.0.30 () {60 vars in 1152 bytes} [Mon May 23 15:40:38 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:40:38.546529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f795d-f425-430d-ba3a-2fb264952195 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.557478 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f795d-f425-430d-ba3a-2fb264952195 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.587157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:38.646073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a32c00-c2e4-48d6-8235-a50b3901afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.646757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a32c00-c2e4-48d6-8235-a50b3901afa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.647041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a32c00-c2e4-48d6-8235-a50b3901afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.647320 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a32c00-c2e4-48d6-8235-a50b3901afa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.648058 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a32c00-c2e4-48d6-8235-a50b3901afa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.650179 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a32c00-c2e4-48d6-8235-a50b3901afa8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.672273 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1419/2873] 10.222.0.30 () {60 vars in 1143 bytes} [Mon May 23 15:40:38 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:40:38.676405 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac7f31-3b61-47ac-98a0-bb4fb530885e admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.688526 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac7f31-3b61-47ac-98a0-bb4fb530885e admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.693255 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74073fc-deb8-4049-9ccb-966b780b7305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.693938 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74073fc-deb8-4049-9ccb-966b780b7305 admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.694302 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74073fc-deb8-4049-9ccb-966b780b7305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.694590 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74073fc-deb8-4049-9ccb-966b780b7305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.695335 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74073fc-deb8-4049-9ccb-966b780b7305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.696991 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74073fc-deb8-4049-9ccb-966b780b7305 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.712366 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1455/2874] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:40:38.714405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a32c00-c2e4-48d6-8235-a50b3901afa8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.725309 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a32c00-c2e4-48d6-8235-a50b3901afa8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.730409 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058cf2d-46a3-4f8f-ac26-3ff0f2bcf2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.731376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058cf2d-46a3-4f8f-ac26-3ff0f2bcf2a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.731646 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058cf2d-46a3-4f8f-ac26-3ff0f2bcf2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.731925 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058cf2d-46a3-4f8f-ac26-3ff0f2bcf2a2 admin admin] PATH_INFO: `/v3/projects/f40b32e4715540068de04b9e051aecfb/users/a9f8fb3d165242559523b575d31dca83/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.749477 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058cf2d-46a3-4f8f-ac26-3ff0f2bcf2a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40b32e4715540068de04b9e051aecfb, user_id=a9f8fb3d165242559523b575d31dca83, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.752494 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058cf2d-46a3-4f8f-ac26-3ff0f2bcf2a2 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.767304 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1420/2875] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:40:38 2022] PUT /identity/v3/projects/f40b32e4715540068de04b9e051aecfb/users/a9f8fb3d165242559523b575d31dca83/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:40:38.771513 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74073fc-deb8-4049-9ccb-966b780b7305 admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.782133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74073fc-deb8-4049-9ccb-966b780b7305 admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.843066 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1181a-4bcf-4574-96e6-73cbca66061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.843713 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1181a-4bcf-4574-96e6-73cbca66061a admin admin] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.844000 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1181a-4bcf-4574-96e6-73cbca66061a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.844278 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1181a-4bcf-4574-96e6-73cbca66061a admin admin] PATH_INFO: `/v3/roles` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.845053 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1181a-4bcf-4574-96e6-73cbca66061a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.846665 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1181a-4bcf-4574-96e6-73cbca66061a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.857869 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1456/2876] 10.222.0.30 () {58 vars in 1121 bytes} [Mon May 23 15:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:40:38.862076 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058cf2d-46a3-4f8f-ac26-3ff0f2bcf2a2 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.874577 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058cf2d-46a3-4f8f-ac26-3ff0f2bcf2a2 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.882101 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36537e2c-4e55-4db0-ac67-dfa7d9c39c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.883678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36537e2c-4e55-4db0-ac67-dfa7d9c39c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.884151 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36537e2c-4e55-4db0-ac67-dfa7d9c39c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.884624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36537e2c-4e55-4db0-ac67-dfa7d9c39c3c admin admin] PATH_INFO: `/v3/projects/f40b32e4715540068de04b9e051aecfb/users/a9f8fb3d165242559523b575d31dca83/roles/2ec004855b6f4bd5a778750ef8fdda50` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.887927 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36537e2c-4e55-4db0-ac67-dfa7d9c39c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40b32e4715540068de04b9e051aecfb, user_id=a9f8fb3d165242559523b575d31dca83, role_id=2ec004855b6f4bd5a778750ef8fdda50)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.890649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36537e2c-4e55-4db0-ac67-dfa7d9c39c3c admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.902747 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1421/2877] 10.222.0.30 () {60 vars in 1482 bytes} [Mon May 23 15:40:38 2022] PUT /identity/v3/projects/f40b32e4715540068de04b9e051aecfb/users/a9f8fb3d165242559523b575d31dca83/roles/2ec004855b6f4bd5a778750ef8fdda50 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:40:43.615531 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baf9ed-3c13-457a-bb19-fbfc10464403 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:43.616005 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baf9ed-3c13-457a-bb19-fbfc10464403 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:43.616121 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baf9ed-3c13-457a-bb19-fbfc10464403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:43.647488 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-f9baf9ed-3c13-457a-bb19-fbfc10464403 None None] MFA Rules not processed for user `a9f8fb3d165242559523b575d31dca83`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:43.648700 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f9baf9ed-3c13-457a-bb19-fbfc10464403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:43.725579 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1457/2878] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:40:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:40:43.741822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36537e2c-4e55-4db0-ac67-dfa7d9c39c3c admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:43.744707 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-36537e2c-4e55-4db0-ac67-dfa7d9c39c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:43.827871 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36537e2c-4e55-4db0-ac67-dfa7d9c39c3c admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:43.834680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88c478-2906-4eb9-beed-59fb9a5c7412 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:43.835377 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88c478-2906-4eb9-beed-59fb9a5c7412 service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:43.835662 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88c478-2906-4eb9-beed-59fb9a5c7412 service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:43.835940 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88c478-2906-4eb9-beed-59fb9a5c7412 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:43.895734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b88c478-2906-4eb9-beed-59fb9a5c7412 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:43.897405 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b88c478-2906-4eb9-beed-59fb9a5c7412 service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:43.907419 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1422/2879] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:40:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:40:44.985297 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39940c-ada8-4663-a5eb-04670334c3ea None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:44.985850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39940c-ada8-4663-a5eb-04670334c3ea None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:44.985850 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39940c-ada8-4663-a5eb-04670334c3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:45.007826 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-cf39940c-ada8-4663-a5eb-04670334c3ea None None] MFA Rules not processed for user `aec46cf4f87241a2a4581e34c154f5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:45.009289 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-cf39940c-ada8-4663-a5eb-04670334c3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:45.016844 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1458/2880] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:40:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:40:49.130363 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df44c4a-fc10-456e-ab6b-660d07b0a9a8 None None] REQUEST_METHOD: `POST` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:49.130950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df44c4a-fc10-456e-ab6b-660d07b0a9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:49.130950 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df44c4a-fc10-456e-ab6b-660d07b0a9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:49.138814 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:49.152116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.auth.core [None req-8df44c4a-fc10-456e-ab6b-660d07b0a9a8 None None] MFA Rules not processed for user `01d7c2fa381541139ea7aebaf2ae688f`. Rule list: `[]` (Enabled: `True`). {{(pid=80421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:49.153668 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8df44c4a-fc10-456e-ab6b-660d07b0a9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:49.209884 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1423/2881] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:40:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:40:49.548299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39940c-ada8-4663-a5eb-04670334c3ea None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:49.549787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-cf39940c-ada8-4663-a5eb-04670334c3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:49.565318 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:49.612705 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39940c-ada8-4663-a5eb-04670334c3ea None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:49.616692 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f775837-2f0e-44ab-9427-ea56a906094d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:49.617392 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f775837-2f0e-44ab-9427-ea56a906094d service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:49.617679 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f775837-2f0e-44ab-9427-ea56a906094d service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:49.617963 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f775837-2f0e-44ab-9427-ea56a906094d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:49.619762 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-4f775837-2f0e-44ab-9427-ea56a906094d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:49.631666 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f775837-2f0e-44ab-9427-ea56a906094d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:49.633131 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f775837-2f0e-44ab-9427-ea56a906094d service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:49.643964 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1459/2882] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:40:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:40:50.141307 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df44c4a-fc10-456e-ab6b-660d07b0a9a8 None None] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:50.142866 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-8df44c4a-fc10-456e-ab6b-660d07b0a9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:50.224123 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df44c4a-fc10-456e-ab6b-660d07b0a9a8 None None] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:50.229121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e96ba-1056-4a9d-a556-a5db500948aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:50.229773 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e96ba-1056-4a9d-a556-a5db500948aa service placement] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:50.230059 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e96ba-1056-4a9d-a556-a5db500948aa service placement] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:50.230339 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e96ba-1056-4a9d-a556-a5db500948aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:50.232769 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-177e96ba-1056-4a9d-a556-a5db500948aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:50.244442 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e96ba-1056-4a9d-a556-a5db500948aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:50.246073 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e96ba-1056-4a9d-a556-a5db500948aa service placement] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:50.257039 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1424/2883] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:40:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:40:52.069465 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f775837-2f0e-44ab-9427-ea56a906094d service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:52.079242 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f775837-2f0e-44ab-9427-ea56a906094d service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:52.083670 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63937db-454b-4063-b88e-259d67646317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:52.084519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63937db-454b-4063-b88e-259d67646317 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:52.084737 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63937db-454b-4063-b88e-259d67646317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:52.085060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63937db-454b-4063-b88e-259d67646317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:52.086959 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-d63937db-454b-4063-b88e-259d67646317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:52.100085 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63937db-454b-4063-b88e-259d67646317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:52.101831 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63937db-454b-4063-b88e-259d67646317 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:52.118644 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1460/2884] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:40:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:40:58.680234 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e96ba-1056-4a9d-a556-a5db500948aa service placement] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:58.692644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e96ba-1056-4a9d-a556-a5db500948aa service placement] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:58.697975 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dc9b6-1a10-48ee-84a6-98d7e55c2a8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:58.698757 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dc9b6-1a10-48ee-84a6-98d7e55c2a8a service nova] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:58.699041 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dc9b6-1a10-48ee-84a6-98d7e55c2a8a service nova] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:58.699315 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dc9b6-1a10-48ee-84a6-98d7e55c2a8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:58.701117 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-3d9dc9b6-1a10-48ee-84a6-98d7e55c2a8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:58.713260 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9dc9b6-1a10-48ee-84a6-98d7e55c2a8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:58.714861 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9dc9b6-1a10-48ee-84a6-98d7e55c2a8a service nova] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:58.725916 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1425/2885] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:40:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:02.114119 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=27 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:03.174332 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63937db-454b-4063-b88e-259d67646317 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:03.187983 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63937db-454b-4063-b88e-259d67646317 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:03.193778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d2a1e-6f86-4959-bed9-bfa5b9334c37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:03.194538 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d2a1e-6f86-4959-bed9-bfa5b9334c37 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:03.194855 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d2a1e-6f86-4959-bed9-bfa5b9334c37 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:03.195127 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d2a1e-6f86-4959-bed9-bfa5b9334c37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:03.196957 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2f3d2a1e-6f86-4959-bed9-bfa5b9334c37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:03.210347 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d2a1e-6f86-4959-bed9-bfa5b9334c37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:03.211941 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d2a1e-6f86-4959-bed9-bfa5b9334c37 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:03.223967 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1461/2886] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:41:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:41:08.721537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=61 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:13.219643 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=6 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:16.740928 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dc9b6-1a10-48ee-84a6-98d7e55c2a8a service nova] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:16.750599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dc9b6-1a10-48ee-84a6-98d7e55c2a8a service nova] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:16.754187 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14f213-bd6a-4e4d-842d-28a6ff0a4ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:16.754818 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14f213-bd6a-4e4d-842d-28a6ff0a4ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:16.755100 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14f213-bd6a-4e4d-842d-28a6ff0a4ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:16.755376 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14f213-bd6a-4e4d-842d-28a6ff0a4ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:16.764204 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14f213-bd6a-4e4d-842d-28a6ff0a4ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:16.765836 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14f213-bd6a-4e4d-842d-28a6ff0a4ff4 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:16.775231 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1426/2887] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:41:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:26.771091 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=6 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:41.275154 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f6758-00fe-407c-a0c7-2eb62d4d2467 None None] REQUEST_METHOD: `POST` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.275671 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f6758-00fe-407c-a0c7-2eb62d4d2467 None None] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.275773 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f6758-00fe-407c-a0c7-2eb62d4d2467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.303074 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.auth.core [None req-b70f6758-00fe-407c-a0c7-2eb62d4d2467 None None] MFA Rules not processed for user `01d7c2fa381541139ea7aebaf2ae688f`. Rule list: `[]` (Enabled: `True`). {{(pid=80422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:41.304269 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b70f6758-00fe-407c-a0c7-2eb62d4d2467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:41.327468 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1462/2888] 10.222.0.30 () {58 vars in 957 bytes} [Mon May 23 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:41.640162 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14f213-bd6a-4e4d-842d-28a6ff0a4ff4 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:41.681075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14f213-bd6a-4e4d-842d-28a6ff0a4ff4 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:41.684681 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd24b4e-b0dc-43d4-96c3-cb2902932d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:41.685350 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd24b4e-b0dc-43d4-96c3-cb2902932d56 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.685628 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd24b4e-b0dc-43d4-96c3-cb2902932d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.685899 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd24b4e-b0dc-43d4-96c3-cb2902932d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.687665 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-dcd24b4e-b0dc-43d4-96c3-cb2902932d56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:41.710930 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd24b4e-b0dc-43d4-96c3-cb2902932d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:41.712335 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd24b4e-b0dc-43d4-96c3-cb2902932d56 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:41.720343 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1427/2889] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:42:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:51.325430 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=15 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:51.716698 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=23 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:29.226084 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f6758-00fe-407c-a0c7-2eb62d4d2467 None None] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:29.227480 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-b70f6758-00fe-407c-a0c7-2eb62d4d2467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:29.270629 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f6758-00fe-407c-a0c7-2eb62d4d2467 None None] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:29.275663 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fad2d-10bc-4bec-a417-51ed037a9dba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:29.276301 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fad2d-10bc-4bec-a417-51ed037a9dba service placement] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:29.276583 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fad2d-10bc-4bec-a417-51ed037a9dba service placement] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:29.276858 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fad2d-10bc-4bec-a417-51ed037a9dba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:29.278615 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-fe0fad2d-10bc-4bec-a417-51ed037a9dba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:29.290829 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fad2d-10bc-4bec-a417-51ed037a9dba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:29.292738 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fad2d-10bc-4bec-a417-51ed037a9dba service placement] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:29.302881 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1463/2890] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:43:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:30.447525 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd24b4e-b0dc-43d4-96c3-cb2902932d56 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:30.456193 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd24b4e-b0dc-43d4-96c3-cb2902932d56 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:30.459878 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5e6b6-4da8-4dda-ac86-4ec2ebc75332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:30.460600 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5e6b6-4da8-4dda-ac86-4ec2ebc75332 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:30.461000 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5e6b6-4da8-4dda-ac86-4ec2ebc75332 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:30.461338 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5e6b6-4da8-4dda-ac86-4ec2ebc75332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:30.463312 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-aec5e6b6-4da8-4dda-ac86-4ec2ebc75332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:30.473734 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5e6b6-4da8-4dda-ac86-4ec2ebc75332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:30.475202 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5e6b6-4da8-4dda-ac86-4ec2ebc75332 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:30.483774 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1428/2891] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:43:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:30.585186 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fad2d-10bc-4bec-a417-51ed037a9dba service placement] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:30.595057 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fad2d-10bc-4bec-a417-51ed037a9dba service placement] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:30.598846 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ba2cb-f779-4f8a-bf35-5496b851a245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:30.599593 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ba2cb-f779-4f8a-bf35-5496b851a245 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:30.600030 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ba2cb-f779-4f8a-bf35-5496b851a245 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:30.600380 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ba2cb-f779-4f8a-bf35-5496b851a245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:30.602183 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-bf2ba2cb-f779-4f8a-bf35-5496b851a245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:30.640250 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ba2cb-f779-4f8a-bf35-5496b851a245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:30.641722 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ba2cb-f779-4f8a-bf35-5496b851a245 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:30.661911 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1464/2892] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:43:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:40.479911 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=6 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:40.659299 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=36 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:26.966028 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5e6b6-4da8-4dda-ac86-4ec2ebc75332 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:26.975511 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5e6b6-4da8-4dda-ac86-4ec2ebc75332 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:26.979156 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d4fbc-495d-4d48-b199-894f3cae6109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:26.979792 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d4fbc-495d-4d48-b199-894f3cae6109 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:26.980078 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d4fbc-495d-4d48-b199-894f3cae6109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:26.980356 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d4fbc-495d-4d48-b199-894f3cae6109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:26.982126 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-5b0d4fbc-495d-4d48-b199-894f3cae6109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:26.993564 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d4fbc-495d-4d48-b199-894f3cae6109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:26.995010 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d4fbc-495d-4d48-b199-894f3cae6109 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:27.004532 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1429/2893] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:44:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:37.000248 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=6 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:40.313133 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ba2cb-f779-4f8a-bf35-5496b851a245 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:40.321892 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ba2cb-f779-4f8a-bf35-5496b851a245 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:40.325503 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:40.326132 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:40.326438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:40.326729 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:40.329620 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:40.416345 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:40.417734 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:40.436742 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1465/2894] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:43.744570 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d4fbc-495d-4d48-b199-894f3cae6109 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:43.756175 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d4fbc-495d-4d48-b199-894f3cae6109 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:43.761543 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d1b24-c5e9-4bb3-852f-d60167322a40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:43.762186 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d1b24-c5e9-4bb3-852f-d60167322a40 service placement] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:43.762473 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d1b24-c5e9-4bb3-852f-d60167322a40 service placement] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:43.762752 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d1b24-c5e9-4bb3-852f-d60167322a40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:43.764624 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b01d1b24-c5e9-4bb3-852f-d60167322a40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:43.776452 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d1b24-c5e9-4bb3-852f-d60167322a40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:43.778219 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d1b24-c5e9-4bb3-852f-d60167322a40 service placement] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:43.790157 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1430/2895] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:44:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:46.664044 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:46.665796 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:46.679412 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11405a1-3097-4cc8-819b-7aac3cd89df5 service neutron] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:46.685222 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:46.686528 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:46.686810 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:46.687113 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] PATH_INFO: `/v3/users/01d7c2fa381541139ea7aebaf2ae688f` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:46.688354 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d7c2fa381541139ea7aebaf2ae688f)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:46.690292 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:46.742310 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d7c2fa381541139ea7aebaf2ae688f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:46.754812 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d7c2fa381541139ea7aebaf2ae688f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:46.763493 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d7c2fa381541139ea7aebaf2ae688f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:46.764300 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d7c2fa381541139ea7aebaf2ae688f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:46.765093 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d7c2fa381541139ea7aebaf2ae688f'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:46.782560 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1466/2896] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:44:46 2022] DELETE /identity/v3/users/01d7c2fa381541139ea7aebaf2ae688f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:46.952761 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d1b24-c5e9-4bb3-852f-d60167322a40 service placement] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:46.963573 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d1b24-c5e9-4bb3-852f-d60167322a40 service placement] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:47.022870 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:47.023724 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:47.024005 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:47.024280 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] PATH_INFO: `/v3/projects/449514f02fbc485484110bde5a2018c0` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:47.025544 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=449514f02fbc485484110bde5a2018c0)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:47.027413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:47.076184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449514f02fbc485484110bde5a2018c0', 'request_id': 'req-d9c43254-04d8-461c-928e-6c003e8a43a8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.081803 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449514f02fbc485484110bde5a2018c0', 'request_id': 'req-d9c43254-04d8-461c-928e-6c003e8a43a8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.088982 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449514f02fbc485484110bde5a2018c0', 'request_id': 'req-d9c43254-04d8-461c-928e-6c003e8a43a8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.100697 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] The token cache is being invalidate because project 449514f02fbc485484110bde5a2018c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:44:47.101028 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.101652 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.103188 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1431/2897] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:44:46 2022] DELETE /identity/v3/projects/449514f02fbc485484110bde5a2018c0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:47.107469 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:47.109349 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:47.122060 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fd2a4-9583-46ca-b65f-69f33ab3326e admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:47.155512 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:47.180221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:47.181126 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:47.181411 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:47.181687 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] PATH_INFO: `/v3/users/5e82d4a31aa64c17a51fd70ae80a8975` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:47.182930 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e82d4a31aa64c17a51fd70ae80a8975)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:47.185572 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:47.230994 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e82d4a31aa64c17a51fd70ae80a8975'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.242890 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e82d4a31aa64c17a51fd70ae80a8975'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.250988 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e82d4a31aa64c17a51fd70ae80a8975'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.252156 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e82d4a31aa64c17a51fd70ae80a8975'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.252783 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e82d4a31aa64c17a51fd70ae80a8975'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.265961 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1467/2898] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:44:47 2022] DELETE /identity/v3/users/5e82d4a31aa64c17a51fd70ae80a8975 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:47.441949 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:47.443504 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:47.459529 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c43254-04d8-461c-928e-6c003e8a43a8 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:47.520822 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:47.521712 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:47.521992 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:47.522266 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] PATH_INFO: `/v3/projects/ff6357e23ce94690840a2e24ec0a6143` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:47.523435 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff6357e23ce94690840a2e24ec0a6143)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:47.525394 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:47.547206 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:47.573170 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff6357e23ce94690840a2e24ec0a6143', 'request_id': 'req-b596431b-8c54-4c60-aa5a-f3c598eb7518'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.578658 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff6357e23ce94690840a2e24ec0a6143', 'request_id': 'req-b596431b-8c54-4c60-aa5a-f3c598eb7518'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.585089 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff6357e23ce94690840a2e24ec0a6143', 'request_id': 'req-b596431b-8c54-4c60-aa5a-f3c598eb7518'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.596781 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] The token cache is being invalidate because project ff6357e23ce94690840a2e24ec0a6143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:44:47.597098 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.597776 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:47.599332 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1432/2899] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:44:47 2022] DELETE /identity/v3/projects/ff6357e23ce94690840a2e24ec0a6143 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:57.262001 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:57.593630 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:03.438244 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:03.439982 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:03.527467 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726a9e2-a0e1-4e71-a996-6703f818616f admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:03.631640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '232cfaefaf584ac59720713e02cf2c2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:03.632678 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:03.632984 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:03.633311 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:03.635198 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:03.648073 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:03.649724 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:03.662518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1468/2900] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:45:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:45:04.802471 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:04.803880 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:04.862596 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596431b-8c54-4c60-aa5a-f3c598eb7518 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:04.866116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d85e20dc884cd0bad5bb6c304776ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:04.866767 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:04.867055 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:04.867334 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:04.869700 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:04.883460 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:04.885037 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:04.896537 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1433/2901] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:45:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:45:13.250062 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:13.252421 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:13.307391 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:13.309121 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:13.353341 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9d4e2-cbbc-4c74-b536-78381a6bf802 service nova] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:13.358918 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c15135-4cc9-4d9b-b2b7-dca9a4cbbc86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:13.359519 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c15135-4cc9-4d9b-b2b7-dca9a4cbbc86 service placement] REQUEST_METHOD: `GET` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:13.359873 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c15135-4cc9-4d9b-b2b7-dca9a4cbbc86 service placement] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:13.360259 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c15135-4cc9-4d9b-b2b7-dca9a4cbbc86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:13.362206 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-24c15135-4cc9-4d9b-b2b7-dca9a4cbbc86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:13.378922 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c15135-4cc9-4d9b-b2b7-dca9a4cbbc86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:13.380539 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c15135-4cc9-4d9b-b2b7-dca9a4cbbc86 service placement] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:13.391752 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1469/2902] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:45:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:45:13.394876 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301b5ba-3bf8-4248-96db-7f93ac8b7e87 service neutron] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:13.399903 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:13.400890 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:13.401184 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:13.401463 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] PATH_INFO: `/v3/users/aec46cf4f87241a2a4581e34c154f5a8` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:13.402664 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec46cf4f87241a2a4581e34c154f5a8)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:13.404599 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:13.453116 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec46cf4f87241a2a4581e34c154f5a8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.453768 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec46cf4f87241a2a4581e34c154f5a8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.460576 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec46cf4f87241a2a4581e34c154f5a8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.461269 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec46cf4f87241a2a4581e34c154f5a8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.473224 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec46cf4f87241a2a4581e34c154f5a8'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.486355 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1434/2903] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:45:13 2022] DELETE /identity/v3/users/aec46cf4f87241a2a4581e34c154f5a8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:45:13.659160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c15135-4cc9-4d9b-b2b7-dca9a4cbbc86 service placement] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:13.673655 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c15135-4cc9-4d9b-b2b7-dca9a4cbbc86 service placement] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:13.684123 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=100 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:13.734556 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:13.735438 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:13.735711 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:13.735987 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] PATH_INFO: `/v3/projects/fe810dde94644f57b3b6262366c8f126` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:13.737195 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe810dde94644f57b3b6262366c8f126)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:13.739151 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:13.785640 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe810dde94644f57b3b6262366c8f126', 'request_id': 'req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.791913 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe810dde94644f57b3b6262366c8f126', 'request_id': 'req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.797747 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe810dde94644f57b3b6262366c8f126', 'request_id': 'req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.810221 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] The token cache is being invalidate because project fe810dde94644f57b3b6262366c8f126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:45:13.810518 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.811150 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.813149 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1470/2904] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:45:13 2022] DELETE /identity/v3/projects/fe810dde94644f57b3b6262366c8f126 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:45:13.817054 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:13.818936 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:13.833155 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88b353-7462-4a96-8999-a7416c7c4d76 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:13.852563 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:13.895812 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:13.896684 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:13.897051 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:13.897328 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] PATH_INFO: `/v3/users/a9f8fb3d165242559523b575d31dca83` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:13.898644 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9f8fb3d165242559523b575d31dca83)` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:13.900842 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:13.949189 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9f8fb3d165242559523b575d31dca83'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.949865 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9f8fb3d165242559523b575d31dca83'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.956649 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9f8fb3d165242559523b575d31dca83'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.957305 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9f8fb3d165242559523b575d31dca83'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.969064 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.notifications [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9f8fb3d165242559523b575d31dca83'} {{(pid=80421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:13.981680 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1435/2905] 10.222.0.30 () {58 vars in 1223 bytes} [Mon May 23 15:45:13 2022] DELETE /identity/v3/users/a9f8fb3d165242559523b575d31dca83 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:45:14.149104 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] Authenticating user token {{(pid=80422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:14.150585 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.fernet_utils [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:14.163673 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b959026-e21a-4d28-9b3d-cf5a767f4f5a admin admin] Validating token access rules against request {{(pid=80422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:14.225981 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61ee072739a04036a34cf038cd44e02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c586cdfef8dc4cea9cf389817d345b08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:14.227499 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:14.227778 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:14.228139 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] PATH_INFO: `/v3/projects/f40b32e4715540068de04b9e051aecfb` {{(pid=80422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:14.229432 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40b32e4715540068de04b9e051aecfb)` {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:14.231377 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] RBAC: Authorization granted {{(pid=80422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:14.280567 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:14.283799 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40b32e4715540068de04b9e051aecfb', 'request_id': 'req-5d003429-229f-495d-b65c-a7a5d3115a49'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:14.293079 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40b32e4715540068de04b9e051aecfb', 'request_id': 'req-5d003429-229f-495d-b65c-a7a5d3115a49'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:14.299664 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40b32e4715540068de04b9e051aecfb', 'request_id': 'req-5d003429-229f-495d-b65c-a7a5d3115a49'} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:14.313212 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] The token cache is being invalidate because project f40b32e4715540068de04b9e051aecfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:45:14.313494 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:14.314160 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG keystone.notifications [None req-5d003429-229f-495d-b65c-a7a5d3115a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:14.316035 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: [pid: 80422|app: 0|req: 1471/2906] 10.222.0.30 () {58 vars in 1232 bytes} [Mon May 23 15:45:14 2022] DELETE /identity/v3/projects/f40b32e4715540068de04b9e051aecfb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:45:23.978141 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:24.309787 dv-cinder-841125-8-iscsi devstack@keystone.service[80422]: DEBUG dbcounter [-] [80422] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:01.831222 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] Authenticating user token {{(pid=80421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:01.832626 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:01.913900 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea55b4-0f92-4f4e-ab6f-fb55606b9204 admin admin] Validating token access rules against request {{(pid=80421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:01.972413 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d797c94-7b99-4c2f-b9c0-14e09fdec45f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a7dd5a0102d4d3493f09e809cb3caaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d84c37c0bf2446aa66ba9a9783c454a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:01.973075 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d797c94-7b99-4c2f-b9c0-14e09fdec45f service placement] REQUEST_METHOD: `GET` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:01.973357 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d797c94-7b99-4c2f-b9c0-14e09fdec45f service placement] SCRIPT_NAME: `/identity` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:01.973627 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d797c94-7b99-4c2f-b9c0-14e09fdec45f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:01.975416 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.fernet_utils [None req-7d797c94-7b99-4c2f-b9c0-14e09fdec45f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:01.986406 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d797c94-7b99-4c2f-b9c0-14e09fdec45f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:01.987994 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d797c94-7b99-4c2f-b9c0-14e09fdec45f service placement] RBAC: Authorization granted {{(pid=80421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:01.997678 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: [pid: 80421|app: 0|req: 1436/2907] 10.222.0.30 () {60 vars in 1365 bytes} [Mon May 23 15:49:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:11.993512 dv-cinder-841125-8-iscsi devstack@keystone.service[80421]: DEBUG dbcounter [-] [80421] Writing DB stats keystone:SELECT=58 {{(pid=80421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}